tf.contrib.kfac.fisher_blocks.FisherBlock

Class FisherBlock

Defined in tensorflow/contrib/kfac/python/ops/fisher_blocks.py.

Abstract base class for objects modeling approximate Fisher matrix blocks.

Subclasses must implement register_matpower, multiply_matpower, instantiate_factors, tensors_to_compute_grads, and num_registered_towers methods.

Properties

num_registered_towers

Number of towers registered for this FisherBlock.

Typically equal to the number of towers in a multi-tower setup.

Methods

__init__

__init__(layer_collection)

instantiate_factors

instantiate_factors(
    grads_list,
    damping
)

Creates and registers the component factors of this Fisher block.

Args:

  • grads_list: A list gradients (each a Tensor or tuple of Tensors) with respect to the tensors returned by tensors_to_compute_grads() that are to be used to estimate the block.
  • damping: The damping factor (float or Tensor).

multiply

multiply(vector)

Multiplies the vector by the (damped) block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.

Returns:

The vector left-multiplied by the (damped) block.

multiply_cholesky

multiply_cholesky(
    vector,
    transpose=False
)

Multiplies the vector by the (damped) Cholesky-factor of the block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.
  • transpose: Bool. If true the Cholesky factor is transposed before multiplying the vector. (Default: False)

Returns:

The vector left-multiplied by the (damped) Cholesky-factor of the block.

multiply_cholesky_inverse

multiply_cholesky_inverse(
    vector,
    transpose=False
)

Multiplies vector by the (damped) inverse Cholesky-factor of the block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.
  • transpose: Bool. If true the Cholesky factor inverse is transposed before multiplying the vector. (Default: False)

Returns:

Vector left-multiplied by (damped) inverse Cholesky-factor of the block.

multiply_inverse

multiply_inverse(vector)

Multiplies the vector by the (damped) inverse of the block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.

Returns:

The vector left-multiplied by the (damped) inverse of the block.

multiply_matpower

multiply_matpower(
    vector,
    exp
)

Multiplies the vector by the (damped) matrix-power of the block.

Args:

  • vector: The vector (a Tensor or tuple of Tensors) to be multiplied.
  • exp: A float representing the power to raise the block by before multiplying it by the vector.

Returns:

The vector left-multiplied by the (damped) matrix-power of the block.

register_cholesky

register_cholesky()

Registers a Cholesky factor to be computed by the block.

register_cholesky_inverse

register_cholesky_inverse()

Registers an inverse Cholesky factor to be computed by the block.

register_inverse

register_inverse()

Registers a matrix inverse to be computed by the block.

register_matpower

register_matpower(exp)

Registers a matrix power to be computed by the block.

Args:

  • exp: A float representing the power to raise the block by.

tensors_to_compute_grads

tensors_to_compute_grads()

Returns the Tensor(s) with respect to which this FisherBlock needs grads.