tf.contrib.kfac.fisher_factors.ConvOutputKroneckerFactor

Class ConvOutputKroneckerFactor

Inherits From: InverseProvidingFactor

Defined in tensorflow/contrib/kfac/python/ops/fisher_factors.py.

Kronecker factor for the output side of a convolutional layer.

Estimates E[ ds ds^T ] where s is the preactivations of a convolutional layer given example x and ds = (d / d s) log(p(y|x, w)). Expectation is taken over all examples and locations.

Equivalent to Gamma in https://arxiv.org/abs/1602.01407 for details. See Section 3.1 Estimating the factors.

Properties

name

Methods

__init__

__init__(
    outputs_grads,
    data_format=None
)

Initializes ConvOutputKroneckerFactor.

Args:

  • outputs_grads: List of list of Tensors. Each Tensor is of shape [batch_size, ..spatial_input_size.., out_channels]. First list index is source, the second is tower.
  • data_format: None or str. Format of outputs_grads.

Raises:

  • ValueError: If channels are not final dimension.

get_cov

get_cov()

get_cov_var

get_cov_var()

Get variable backing this FisherFactor.

May or may not be the same as self.get_cov()

Returns:

Variable of shape self._cov_shape.

get_eigendecomp

get_eigendecomp()

Creates or retrieves eigendecomposition of self._cov.

get_inverse

get_inverse(damping_func)

get_matpower

get_matpower(
    exp,
    damping_func
)

instantiate_cov_variables

instantiate_cov_variables()

Makes the internal cov variable(s).

instantiate_inv_variables

instantiate_inv_variables()

Makes the internal "inverse" variable(s).

left_multiply_matpower

left_multiply_matpower(
    x,
    exp,
    damping_func
)

make_covariance_update_op

make_covariance_update_op(ema_decay)

Constructs and returns the covariance update Op.

Args:

  • ema_decay: The exponential moving average decay (float or Tensor).

Returns:

An Op for updating the covariance Variable referenced by _cov.

make_inverse_update_ops

make_inverse_update_ops()

Create and return update ops corresponding to registered computations.

register_inverse

register_inverse(damping_func)

register_matpower

register_matpower(
    exp,
    damping_func
)

Registers a matrix power to be maintained and served on demand.

This creates a variable and signals make_inverse_update_ops to make the corresponding update op. The variable can be read via the method get_matpower.

Args:

  • exp: float. The exponent to use in the matrix power.
  • damping_func: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().

right_multiply_matpower

right_multiply_matpower(
    x,
    exp,
    damping_func
)