tf.contrib.kfac.fisher_blocks.FisherBlock

Class FisherBlock

Defined in tensorflow/contrib/kfac/python/ops/fisher_blocks.py.

Abstract base class for objects modeling approximate Fisher matrix blocks.

Subclasses must implement multiply_inverse(), instantiate_factors(), and tensors_to_compute_grads() methods.

Properties

num_registered_minibatches

Number of minibatches registered for this FisherBlock.

Typically equal to the number of towers in a multi-tower setup.

Methods

__init__

__init__(layer_collection)

instantiate_factors

instantiate_factors(
    grads_list,
    damping
)

Creates and registers the component factors of this Fisher block.

Args:

  • grads_list: A list gradients (each a Tensor or tuple of Tensors) with respect to the tensors returned by tensors_to_compute_grads() that are to be used to estimate the block.
  • damping: The damping factor (float or Tensor).

multiply

multiply(vector)

Multiplies the vector by the (damped) block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.

Returns:

The vector left-multiplied by the (damped) block.

multiply_inverse

multiply_inverse(vector)

Multiplies the vector by the (damped) inverse of the block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.

Returns:

The vector left-multiplied by the (damped) inverse of the block.

tensors_to_compute_grads

tensors_to_compute_grads()

Returns the Tensor(s) with respect to which this FisherBlock needs grads.