tf.contrib.kfac.loss_functions.CategoricalLogitsNegativeLogProbLoss

Class CategoricalLogitsNegativeLogProbLoss

Inherits From: DistributionNegativeLogProbLoss, NaturalParamsNegativeLogProbLoss

Defined in tensorflow/contrib/kfac/python/ops/loss_functions.py.

Neg log prob loss for a categorical distribution parameterized by logits.

Note that the Fisher (for a single case) of a categorical distribution, with respect to the natural parameters (i.e. the logits), is given by:

F = diag(p) - p*p^T

where p = softmax(logits). F can be factorized as F = B * B^T where

B = diag(q) - p*q^T

where q is the entry-wise square root of p. This is easy to verify using the fact that q^T*q = 1.

Properties

dist

fisher_factor_inner_shape

fisher_factor_inner_static_shape

hessian_factor_inner_shape

hessian_factor_inner_static_shape

input_minibatches

inputs

num_registered_minibatches

Number of minibatches registered for this LossFunction.

Typically equal to the number of towers in a multi-tower setup.

Returns:

An int representing the number of registered minibatches.

params

targets

Methods

__init__

__init__(
    logits,
    targets=None,
    seed=None
)

Instantiates a CategoricalLogitsNegativeLogProbLoss.

Args:

  • logits: Tensor of shape [batch_size, output_size]. Parameters for underlying distribution.
  • targets: None or Tensor of shape [output_size]. Each elements contains an index in [0, output_size).
  • seed: int or None. Default random seed when sampling.

evaluate

evaluate()

Evaluate the loss function on the targets.

evaluate_on_sample

evaluate_on_sample(seed=None)

Evaluates the log probability on a random sample.

Args:

  • seed: int or None. Random seed for this draw from the distribution.

Returns:

Log probability of sampled targets, summed across examples.

multiply_fisher

multiply_fisher(vector)

multiply_fisher_factor

multiply_fisher_factor(vector)

multiply_fisher_factor_replicated_one_hot

multiply_fisher_factor_replicated_one_hot(index)

multiply_fisher_factor_transpose

multiply_fisher_factor_transpose(vector)

multiply_hessian

multiply_hessian(vector)

multiply_hessian_factor

multiply_hessian_factor(vector)

multiply_hessian_factor_replicated_one_hot

multiply_hessian_factor_replicated_one_hot(index)

multiply_hessian_factor_transpose

multiply_hessian_factor_transpose(vector)

register_additional_minibatch

register_additional_minibatch(
    logits,
    targets=None
)

Register an additiona minibatch's worth of parameters.

Args:

  • logits: Tensor of shape [batch_size, output_size]. Parameters for underlying distribution.
  • targets: None or Tensor of shape [batch_size, output_size]. Each row must be a one-hot vector.

sample

sample(seed)