TensorFlow 2.0 RC is available

# tfp.distributions.VectorDiffeomixture

## Class `VectorDiffeomixture`

VectorDiffeomixture distribution.

Inherits From: `Distribution`

A vector diffeomixture (VDM) is a distribution parameterized by a convex combination of `K` component `loc` vectors, `loc[k], k = 0,...,K-1`, and `K` `scale` matrices `scale[k], k = 0,..., K-1`. It approximates the following compound distribution

``````p(x) = int p(x | z) p(z) dz,
where z is in the K-simplex, and
p(x | z) := p(x | loc=sum_k z[k] loc[k], scale=sum_k z[k] scale[k])
``````

The integral `int p(x | z) p(z) dz` is approximated with a quadrature scheme adapted to the mixture density `p(z)`. The `N` quadrature points `z_{N, n}` and weights `w_{N, n}` (which are non-negative and sum to 1) are chosen such that

`q_N(x) := sum_{n=1}^N w_{n, N} p(x | z_{N, n}) --> p(x)`

as `N --> infinity`.

Since `q_N(x)` is in fact a mixture (of `N` points), we may sample from `q_N` exactly. It is important to note that the VDM is defined as `q_N` above, and not `p(x)`. Therefore, sampling and pdf may be implemented as exact (up to floating point error) methods.

A common choice for the conditional `p(x | z)` is a multivariate Normal.

The implemented marginal `p(z)` is the `SoftmaxNormal`, which is a `K-1` dimensional Normal transformed by a `SoftmaxCentered` bijector, making it a density on the `K`-simplex. That is,

``````Z = SoftmaxCentered(X),
X = Normal(mix_loc / temperature, 1 / temperature)
``````

The default quadrature scheme chooses `z_{N, n}` as `N` midpoints of the quantiles of `p(z)` (generalized quantiles if `K > 2`).

See [Dillon and Langmore (2018)] for more details.

#### About `Vector` distributions in TensorFlow.

The `VectorDiffeomixture` is a non-standard distribution that has properties particularly useful in variational Bayesian methods.

Conditioned on a draw from the SoftmaxNormal, `X|z` is a vector whose components are linear combinations of affine transformations, thus is itself an affine transformation.

#### About `Diffeomixture`s and reparameterization.

The `VectorDiffeomixture` is designed to be reparameterized, i.e., its parameters are only used to transform samples from a distribution which has no trainable parameters. This property is important because backprop stops at sources of stochasticity. That is, as long as the parameters are used after the underlying source of stochasticity, the computed gradient is accurate.

Reparametrization means that we can use gradient-descent (via backprop) to optimize Monte-Carlo objectives. Such objectives are a finite-sample approximation of an expectation and arise throughout scientific computing.

WARNING: If you backprop through a VectorDiffeomixture sample and the "base" distribution is both: not `FULLY_REPARAMETERIZED` and a function of trainable variables, then the gradient is not guaranteed correct!

#### Examples

``````tfd = tfp.distributions

# Create two batches of VectorDiffeomixtures, one with mix_loc=[0.],
# another with mix_loc=. In both cases, `K=2` and the affine
# transformations involve:
# k=0: loc=zeros(dims)  scale=LinearOperatorScaledIdentity
# k=1: loc=[2.]*dims    scale=LinOpDiag
dims = 5
vdm = tfd.VectorDiffeomixture(
mix_loc=[[0.], ],
temperature=[1.],
distribution=tfd.Normal(loc=0., scale=1.),
loc=[
None,  # Equivalent to `np.zeros(dims, dtype=np.float32)`.
np.float32([2.]*dims),
],
scale=[
tf.linalg.LinearOperatorScaledIdentity(
num_rows=dims,
multiplier=np.float32(1.1),
is_positive_definite=True),
tf.linalg.LinearOperatorDiag(
diag=np.linspace(2.5, 3.5, dims, dtype=np.float32),
is_positive_definite=True),
],
validate_args=True)
``````

: Joshua Dillon and Ian Langmore. Quadrature Compound: An approximating family of distributions. arXiv preprint arXiv:1801.03080, 2018. https://arxiv.org/abs/1801.03080

## `__init__`

``````__init__(
mix_loc,
temperature,
distribution,
loc=None,
scale=None,
quadrature_size=8,
quadrature_fn=tfp.distributions.quadrature_scheme_softmaxnormal_quantiles,
validate_args=False,
allow_nan_stats=True,
name='VectorDiffeomixture'
)
``````

Constructs the VectorDiffeomixture on `R^d`.

The vector diffeomixture (VDM) approximates the compound distribution

``````p(x) = int p(x | z) p(z) dz,
where z is in the K-simplex, and
p(x | z) := p(x | loc=sum_k z[k] loc[k], scale=sum_k z[k] scale[k])
``````

#### Args:

• `mix_loc`: `float`-like `Tensor` with shape `[b1, ..., bB, K-1]`. In terms of samples, larger `mix_loc[..., k]` ==> `Z` is more likely to put more weight on its `kth` component.
• `temperature`: `float`-like `Tensor`. Broadcastable with `mix_loc`. In terms of samples, smaller `temperature` means one component is more likely to dominate. I.e., smaller `temperature` makes the VDM look more like a standard mixture of `K` components.
• `distribution`: `tfp.distributions.Distribution`-like instance. Distribution from which `d` iid samples are used as input to the selected affine transformation. Must be a scalar-batch, scalar-event distribution. Typically `distribution.reparameterization_type = FULLY_REPARAMETERIZED` or it is a function of non-trainable parameters. WARNING: If you backprop through a VectorDiffeomixture sample and the `distribution` is not `FULLY_REPARAMETERIZED` yet is a function of trainable variables, then the gradient will be incorrect!
• `loc`: Length-`K` list of `float`-type `Tensor`s. The `k`-th element represents the `shift` used for the `k`-th affine transformation. If the `k`-th item is `None`, `loc` is implicitly `0`. When specified, must have shape `[B1, ..., Bb, d]` where `b >= 0` and `d` is the event size.
• `scale`: Length-`K` list of `LinearOperator`s. Each should be positive-definite and operate on a `d`-dimensional vector space. The `k`-th element represents the `scale` used for the `k`-th affine transformation. `LinearOperator`s must have shape `[B1, ..., Bb, d, d]`, `b >= 0`, i.e., characterizes `b`-batches of `d x d` matrices
• `quadrature_size`: Python `int` scalar representing number of quadrature points. Larger `quadrature_size` means `q_N(x)` better approximates `p(x)`.
• `quadrature_fn`: Python callable taking `normal_loc`, `normal_scale`, `quadrature_size`, `validate_args` and returning `tuple(grid, probs)` representing the SoftmaxNormal grid and corresponding normalized weight. normalized) weight. Default value: `quadrature_scheme_softmaxnormal_quantiles`.
• `validate_args`: Python `bool`, default `False`. When `True` distribution parameters are checked for validity despite possibly degrading runtime performance. When `False` invalid inputs may silently render incorrect outputs.
• `allow_nan_stats`: Python `bool`, default `True`. When `True`, statistics (e.g., mean, mode, variance) use the value "`NaN`" to indicate the result is undefined. When `False`, an exception is raised if one or more of the statistic's batch members are undefined.
• `name`: Python `str` name prefixed to Ops created by this class.

#### Raises:

• `ValueError`: if `not scale or len(scale) < 2`.
• `ValueError`: if `len(loc) != len(scale)`
• `ValueError`: if `quadrature_grid_and_probs is not None` and `len(quadrature_grid_and_probs) != len(quadrature_grid_and_probs)`
• `ValueError`: if `validate_args` and any not scale.is_positive_definite.
• `TypeError`: if any scale.dtype != scale.dtype.
• `TypeError`: if any loc.dtype != scale.dtype.
• `NotImplementedError`: if `len(scale) != 2`.
• `ValueError`: if `not distribution.is_scalar_batch`.
• `ValueError`: if `not distribution.is_scalar_event`.

## Properties

### `allow_nan_stats`

Python `bool` describing behavior when a stat is undefined.

Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.

#### Returns:

• `allow_nan_stats`: Python `bool`.

### `batch_shape`

Shape of a single sample from a single event index as a `TensorShape`.

May be partially defined or unknown.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

#### Returns:

• `batch_shape`: `TensorShape`, possibly unknown.

### `distribution`

Base scalar-event, scalar-batch distribution.

### `dtype`

The `DType` of `Tensor`s handled by this `Distribution`.

### `endpoint_affine`

Affine transformation for each of `K` components.

### `event_shape`

Shape of a single sample from a single batch as a `TensorShape`.

May be partially defined or unknown.

#### Returns:

• `event_shape`: `TensorShape`, possibly unknown.

### `grid`

Grid of mixing probabilities, one for each grid point.

### `interpolated_affine`

Affine transformation for each convex combination of `K` components.

### `mixture_distribution`

Distribution used to select a convex combination of affine transforms.

### `name`

Name prepended to all ops created by this `Distribution`.

### `parameters`

Dictionary of parameters used to instantiate this `Distribution`.

### `reparameterization_type`

Describes how samples from the distribution are reparameterized.

Currently this is one of the static instances `tfd.FULLY_REPARAMETERIZED` or `tfd.NOT_REPARAMETERIZED`.

#### Returns:

An instance of `ReparameterizationType`.

### `validate_args`

Python `bool` indicating possibly expensive checks are enabled.

## Methods

### `__getitem__`

``````__getitem__(slices)
``````

Slices the batch axes of this distribution, returning a new instance.

``````b = tfd.Bernoulli(logits=tf.zeros([3, 5, 7, 9]))
b.batch_shape  # => [3, 5, 7, 9]
b2 = b[:, tf.newaxis, ..., -2:, 1::2]
b2.batch_shape  # => [3, 1, 5, 2, 4]

x = tf.random.normal([5, 3, 2, 2])
cov = tf.matmul(x, x, transpose_b=True)
chol = tf.cholesky(cov)
loc = tf.random.normal([4, 1, 3, 1])
mvn = tfd.MultivariateNormalTriL(loc, chol)
mvn.batch_shape  # => [4, 5, 3]
mvn.event_shape  # => 
mvn2 = mvn[:, 3:, ..., ::-1, tf.newaxis]
mvn2.batch_shape  # => [4, 2, 3, 1]
mvn2.event_shape  # => 
``````

#### Args:

• `slices`: slices from the [] operator

#### Returns:

• `dist`: A new `tfd.Distribution` instance with sliced parameters.

### `__iter__`

``````__iter__()
``````

### `batch_shape_tensor`

``````batch_shape_tensor(name='batch_shape_tensor')
``````

Shape of a single sample from a single event index as a 1-D `Tensor`.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

#### Args:

• `name`: name to give to the op

#### Returns:

• `batch_shape`: `Tensor`.

### `cdf`

``````cdf(
value,
name='cdf',
**kwargs
)
``````

Cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

``````cdf(x) := P[X <= x]
``````

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `cdf`: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `copy`

``````copy(**override_parameters_kwargs)
``````

Creates a deep copy of the distribution.

#### Args:

• `**override_parameters_kwargs`: String/value dictionary of initialization arguments to override with new values.

#### Returns:

• `distribution`: A new instance of `type(self)` initialized from the union of self.parameters and override_parameters_kwargs, i.e., `dict(self.parameters, **override_parameters_kwargs)`.

### `covariance`

``````covariance(
name='covariance',
**kwargs
)
``````

Covariance.

Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-`k`, vector-valued distribution, it is calculated as,

``````Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]
``````

where `Cov` is a (batch of) `k x k` matrix, `0 <= (i, j) < k`, and `E` denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), `Covariance` shall return a (batch of) matrices under some vectorization of the events, i.e.,

``````Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]
``````

where `Cov` is a (batch of) `k' x k'` matrices, `0 <= (i, j) < k' = reduce_prod(event_shape)`, and `Vec` is some function mapping indices of this distribution's event dimensions to indices of a length-`k'` vector.

#### Args:

• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `covariance`: Floating-point `Tensor` with shape `[B1, ..., Bn, k', k']` where the first `n` dimensions are batch coordinates and `k' = reduce_prod(self.event_shape)`.

### `cross_entropy`

``````cross_entropy(
other,
name='cross_entropy'
)
``````

Computes the (Shannon) cross entropy.

Denote this distribution (`self`) by `P` and the `other` distribution by `Q`. Assuming `P, Q` are absolutely continuous with respect to one another and permit densities `p(x) dr(x)` and `q(x) dr(x)`, (Shannon) cross entropy is defined as:

``````H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)
``````

where `F` denotes the support of the random variable `X ~ P`.

#### Returns:

• `cross_entropy`: `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of (Shannon) cross entropy.

### `entropy`

``````entropy(
name='entropy',
**kwargs
)
``````

Shannon entropy in nats.

### `event_shape_tensor`

``````event_shape_tensor(name='event_shape_tensor')
``````

Shape of a single sample from a single batch as a 1-D int32 `Tensor`.

#### Args:

• `name`: name to give to the op

#### Returns:

• `event_shape`: `Tensor`.

### `is_scalar_batch`

``````is_scalar_batch(name='is_scalar_batch')
``````

Indicates that `batch_shape == []`.

#### Args:

• `name`: Python `str` prepended to names of ops created by this function.

#### Returns:

• `is_scalar_batch`: `bool` scalar `Tensor`.

### `is_scalar_event`

``````is_scalar_event(name='is_scalar_event')
``````

Indicates that `event_shape == []`.

#### Args:

• `name`: Python `str` prepended to names of ops created by this function.

#### Returns:

• `is_scalar_event`: `bool` scalar `Tensor`.

### `kl_divergence`

``````kl_divergence(
other,
name='kl_divergence'
)
``````

Computes the Kullback--Leibler divergence.

Denote this distribution (`self`) by `p` and the `other` distribution by `q`. Assuming `p, q` are absolutely continuous with respect to reference measure `r`, the KL divergence is defined as:

``````KL[p, q] = E_p[log(p(X)/q(X))]
= -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x)
= H[p, q] - H[p]
``````

where `F` denotes the support of the random variable `X ~ p`, `H[., .]` denotes (Shannon) cross entropy, and `H[.]` denotes (Shannon) entropy.

#### Returns:

• `kl_divergence`: `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of the Kullback-Leibler divergence.

### `log_cdf`

``````log_cdf(
value,
name='log_cdf',
**kwargs
)
``````

Log cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

``````log_cdf(x) := Log[ P[X <= x] ]
``````

Often, a numerical approximation can be used for `log_cdf(x)` that yields a more accurate answer than simply taking the logarithm of the `cdf` when `x << -1`.

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `logcdf`: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `log_prob`

``````log_prob(
value,
name='log_prob',
**kwargs
)
``````

Log probability density/mass function.

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `log_prob`: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `log_survival_function`

``````log_survival_function(
value,
name='log_survival_function',
**kwargs
)
``````

Log survival function.

Given random variable `X`, the survival function is defined:

``````log_survival_function(x) = Log[ P[X > x] ]
= Log[ 1 - P[X <= x] ]
= Log[ 1 - cdf(x) ]
``````

Typically, different numerical approximations can be used for the log survival function, which are more accurate than `1 - cdf(x)` when `x >> 1`.

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

`Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `mean`

``````mean(
name='mean',
**kwargs
)
``````

Mean.

### `mode`

``````mode(
name='mode',
**kwargs
)
``````

Mode.

### `param_shapes`

``````param_shapes(
cls,
sample_shape,
name='DistributionParamShapes'
)
``````

Shapes of parameters given the desired shape of a call to `sample()`.

This is a class method that describes what key/value arguments are required to instantiate the given `Distribution` so that a particular shape is returned for that instance's call to `sample()`.

Subclasses should override class method `_param_shapes`.

#### Args:

• `sample_shape`: `Tensor` or python list/tuple. Desired shape of a call to `sample()`.
• `name`: name to prepend ops with.

#### Returns:

`dict` of parameter name to `Tensor` shapes.

### `param_static_shapes`

``````param_static_shapes(
cls,
sample_shape
)
``````

param_shapes with static (i.e. `TensorShape`) shapes.

This is a class method that describes what key/value arguments are required to instantiate the given `Distribution` so that a particular shape is returned for that instance's call to `sample()`. Assumes that the sample's shape is known statically.

Subclasses should override class method `_param_shapes` to return constant-valued tensors when constant values are fed.

#### Args:

• `sample_shape`: `TensorShape` or python list/tuple. Desired shape of a call to `sample()`.

#### Returns:

`dict` of parameter name to `TensorShape`.

#### Raises:

• `ValueError`: if `sample_shape` is a `TensorShape` and is not fully defined.

### `prob`

``````prob(
value,
name='prob',
**kwargs
)
``````

Probability density/mass function.

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `prob`: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `quantile`

``````quantile(
value,
name='quantile',
**kwargs
)
``````

Quantile function. Aka "inverse cdf" or "percent point function".

Given random variable `X` and `p in [0, 1]`, the `quantile` is:

``````quantile(p) := x such that P[X <= x] == p
``````

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `quantile`: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `sample`

``````sample(
sample_shape=(),
seed=None,
name='sample',
**kwargs
)
``````

Generate samples of the specified shape.

Note that a call to `sample()` without arguments will generate a single sample.

#### Args:

• `sample_shape`: 0D or 1D `int32` `Tensor`. Shape of the generated samples.
• `seed`: Python integer seed for RNG
• `name`: name to give to the op.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `samples`: a `Tensor` with prepended dimensions `sample_shape`.

### `stddev`

``````stddev(
name='stddev',
**kwargs
)
``````

Standard deviation.

Standard deviation is defined as,

``````stddev = E[(X - E[X])**2]**0.5
``````

where `X` is the random variable associated with this distribution, `E` denotes expectation, and `stddev.shape = batch_shape + event_shape`.

#### Args:

• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `stddev`: Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`.

### `survival_function`

``````survival_function(
value,
name='survival_function',
**kwargs
)
``````

Survival function.

Given random variable `X`, the survival function is defined:

``````survival_function(x) = P[X > x]
= 1 - P[X <= x]
= 1 - cdf(x).
``````

#### Args:

• `value`: `float` or `double` `Tensor`.
• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

`Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `variance`

``````variance(
name='variance',
**kwargs
)
``````

Variance.

Variance is defined as,

``````Var = E[(X - E[X])**2]
``````

where `X` is the random variable associated with this distribution, `E` denotes expectation, and `Var.shape = batch_shape + event_shape`.

#### Args:

• `name`: Python `str` prepended to names of ops created by this function.
• `**kwargs`: Named arguments forwarded to subclass implementation.

#### Returns:

• `variance`: Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`.