Class NaiveDiagonalFB
Defined in tensorflow/contrib/kfac/python/ops/fisher_blocks.py.
FisherBlock using a diagonal matrix approximation.
This type of approximation is generically applicable but quite primitive.
Note that this uses the naive "square the sum estimator", and so is applicable to any type of parameter in principle, but has very high variance.
Properties
num_registered_towers
Methods
__init__
__init__(
layer_collection,
params
)
Creates a NaiveDiagonalFB block.
Args:
layer_collection: The collection of all layers in the K-FAC approximate Fisher information matrix to which this FisherBlock belongs.params: The parameters of this layer (Tensor or tuple of Tensors).
full_fisher_block
full_fisher_block()
instantiate_factors
instantiate_factors(
grads_list,
damping
)
multiply
multiply(vector)
Multiplies the vector by the (damped) block.
Args:
vector: The vector (a Tensor or tuple of Tensors) to be multiplied.
Returns:
The vector left-multiplied by the (damped) block.
multiply_cholesky
multiply_cholesky(
vector,
transpose=False
)
multiply_cholesky_inverse
multiply_cholesky_inverse(
vector,
transpose=False
)
multiply_inverse
multiply_inverse(vector)
Multiplies the vector by the (damped) inverse of the block.
Args:
vector: The vector (a Tensor or tuple of Tensors) to be multiplied.
Returns:
The vector left-multiplied by the (damped) inverse of the block.
multiply_matpower
multiply_matpower(
vector,
exp
)
register_additional_tower
register_additional_tower(batch_size)
Register an additional tower.
Args:
batch_size: The batch size, used in the covariance estimator.
register_cholesky
register_cholesky()
register_cholesky_inverse
register_cholesky_inverse()
register_inverse
register_inverse()
Registers a matrix inverse to be computed by the block.
register_matpower
register_matpower(exp)
tensors_to_compute_grads
tensors_to_compute_grads()