tf.contrib.kfac.fisher_factors.ConvInputKroneckerFactor

Class ConvInputKroneckerFactor

Inherits From: InverseProvidingFactor

Defined in tensorflow/contrib/kfac/python/ops/fisher_factors.py.

Kronecker factor for the input side of a convolutional layer.

Estimates E[ a a^T ] where a is the inputs to a convolutional layer given example x. Expectation is taken over all examples and locations.

Equivalent to Omega in https://arxiv.org/abs/1602.01407 for details. See Section 3.1 Estimating the factors.

Methods

__init__

__init__(
    inputs,
    filter_shape,
    strides,
    padding,
    has_bias=False
)

Initializes ConvInputKroneckerFactor.

Args:

  • inputs: Tensor of shape [batch_size, height, width, in_channels]. Inputs to layer.
  • filter_shape: 1-D Tensor of length 4. Contains [kernel_height, kernel_width, in_channels, out_channels].
  • strides: 1-D Tensor of length 4. Contains [batch_stride, height_stride, width_stride, in_channel_stride].
  • padding: str. Padding method for layer. "SAME" or "VALID".
  • has_bias: bool. If True, append 1 to in_channel.

get_cov

get_cov()

get_cov_var

get_cov_var()

Get variable backing this FisherFactor.

May or may not be the same as self.get_cov()

Returns:

Variable of shape self._cov_shape.

get_damped_inverse

get_damped_inverse(damping)

get_eigendecomp

get_eigendecomp()

Creates or retrieves eigendecomposition of self._cov.

get_matpower

get_matpower(
    exp,
    damping
)

instantiate_covariance

instantiate_covariance()

Instantiates the covariance Variable as the instance member _cov.

left_multiply

left_multiply(
    x,
    damping
)

left_multiply_inverse

left_multiply_inverse(
    x,
    damping
)

make_covariance_update_op

make_covariance_update_op(ema_decay)

Constructs and returns the covariance update Op.

Args:

  • ema_decay: The exponential moving average decay (float or Tensor).

Returns:

An Op for updating the covariance Variable referenced by _cov.

make_inverse_update_ops

make_inverse_update_ops()

Create and return update ops corresponding to registered computations.

register_damped_inverse

register_damped_inverse(damping)

Registers a damped inverse needed by a FisherBlock.

This creates a variable and signals make_inverse_update_ops to make the corresponding update op. The variable can be read via the method get_inverse.

Args:

  • damping: The damping value (float or Tensor) for this factor.

register_matpower

register_matpower(
    exp,
    damping
)

Registers a matrix power needed by a FisherBlock.

This creates a variable and signals make_inverse_update_ops to make the corresponding update op. The variable can be read via the method get_matpower.

Args:

  • exp: The exponent (float or Tensor) to raise the matrix to.
  • damping: The damping value (float or Tensor).

reset_eigendecomp

reset_eigendecomp()

right_multiply

right_multiply(
    x,
    damping
)

right_multiply_inverse

right_multiply_inverse(
    x,
    damping
)