tf.contrib.kfac.fisher_blocks.NaiveDiagonalFB

Class NaiveDiagonalFB

Inherits From: FisherBlock

Defined in tensorflow/contrib/kfac/python/ops/fisher_blocks.py.

FisherBlock using a diagonal matrix approximation.

This type of approximation is generically applicable but quite primitive.

Note that this uses the naive "square the sum estimator", and so is applicable to any type of parameter in principle, but has very high variance.

Properties

num_registered_minibatches

Methods

__init__

__init__(
    layer_collection,
    params
)

Creates a NaiveDiagonalFB block.

Args:

  • layer_collection: The collection of all layers in the K-FAC approximate Fisher information matrix to which this FisherBlock belongs.
  • params: The parameters of this layer (Tensor or tuple of Tensors).

full_fisher_block

full_fisher_block()

instantiate_factors

instantiate_factors(
    grads_list,
    damping
)

multiply

multiply(vector)

multiply_inverse

multiply_inverse(vector)

register_additional_minibatch

register_additional_minibatch(batch_size)

Register an additional minibatch.

Args:

  • batch_size: The batch size, used in the covariance estimator.

tensors_to_compute_grads

tensors_to_compute_grads()