Class NaiveDiagonalFB
Inherits From: FisherBlock
Defined in tensorflow/contrib/kfac/python/ops/fisher_blocks.py.
FisherBlock using a diagonal matrix approximation.
This type of approximation is generically applicable but quite primitive.
Note that this uses the naive "square the sum estimator", and so is applicable to any type of parameter in principle, but has very high variance.
Properties
num_registered_minibatches
Methods
__init__
__init__(
layer_collection,
params
)
Creates a NaiveDiagonalFB block.
Args:
layer_collection: The collection of all layers in the K-FAC approximate Fisher information matrix to which this FisherBlock belongs.params: The parameters of this layer (Tensor or tuple of Tensors).
full_fisher_block
full_fisher_block()
instantiate_factors
instantiate_factors(
grads_list,
damping
)
multiply
multiply(vector)
multiply_inverse
multiply_inverse(vector)
register_additional_minibatch
register_additional_minibatch(batch_size)
Register an additional minibatch.
Args:
batch_size: The batch size, used in the covariance estimator.
tensors_to_compute_grads
tensors_to_compute_grads()