tf.contrib.kfac.fisher_factors.FisherFactor

Class FisherFactor

Defined in tensorflow/contrib/kfac/python/ops/fisher_factors.py.

Base class for objects modeling factors of approximate Fisher blocks.

A FisherFactor represents part of an approximate Fisher Information matrix. For example, one approximation to the Fisher uses the Kronecker product of two FisherFactors A and B, F = kron(A, B). FisherFactors are composed with FisherBlocks to construct a block-diagonal approximation to the full Fisher.

FisherFactors are backed by a single, non-trainable variable that is updated by running FisherFactor.make_covariance_update_op(). The shape and type of this variable is implementation specific.

Note that for blocks that aren't based on approximations, a 'factor' can be the entire block itself, as is the case for the diagonal and full representations.

Properties

name

Methods

__init__

__init__()

get_cholesky

get_cholesky(damping_func)

get_cholesky_inverse

get_cholesky_inverse(damping_func)

get_cov

get_cov()

get_cov_as_linear_operator

get_cov_as_linear_operator()

get_matpower

get_matpower(
    exp,
    damping_func
)

instantiate_cov_variables

instantiate_cov_variables()

Makes the internal cov variable(s).

instantiate_inv_variables

instantiate_inv_variables()

Makes the internal "inverse" variable(s).

make_covariance_update_op

make_covariance_update_op(ema_decay)

Constructs and returns the covariance update Op.

Args:

  • ema_decay: The exponential moving average decay (float or Tensor).

Returns:

An Op for updating the covariance Variable referenced by _cov.

make_inverse_update_ops

make_inverse_update_ops()

Create and return update ops corresponding to registered computations.

register_cholesky

register_cholesky(damping_func)

register_cholesky_inverse

register_cholesky_inverse(damping_func)

register_matpower

register_matpower(
    exp,
    damping_func
)