Google I/O is a wrap! Catch up on TensorFlow sessions

# tf.contrib.distributions.VectorExponentialDiag

The vectorization of the Exponential distribution on `R^k`.

The vector exponential distribution is defined over a subset of `R^k`, and parameterized by a (batch of) length-`k` `loc` vector and a (batch of) `k x k` `scale` matrix: `covariance = scale @ scale.T`, where `@` denotes matrix-multiplication.

#### Mathematical Details

The probability density function (pdf) is defined over the image of the `scale` matrix + `loc`, applied to the positive half-space: `Supp = {loc + scale @ x : x in R^k, x_1 > 0, ..., x_k > 0}`. On this set,

``````pdf(y; loc, scale) = exp(-||x||_1) / Z,  for y in Supp
x = inv(scale) @ (y - loc),
Z = |det(scale)|,
``````

where:

• `loc` is a vector in `R^k`,
• `scale` is a linear operator in `R^{k x k}`, `cov = scale @ scale.T`,
• `Z` denotes the normalization constant, and,
• `||x||_1` denotes the `l1` norm of `x`, `sum_i |x_i|`.

The VectorExponential distribution is a member of the location-scale family, i.e., it can be constructed as,

``````X = (X_1, ..., X_k), each X_i ~ Exponential(rate=1)
Y = (Y_1, ...,Y_k) = scale @ X + loc
``````

#### About `VectorExponential` and `Vector` distributions in TensorFlow.

The `VectorExponential` is a non-standard distribution that has useful properties.

The marginals `Y_1, ..., Y_k` are not Exponential random variables, due to the fact that the sum of Exponential random variables is not Exponential.

Instead, `Y` is a vector whose components are linear combinations of Exponential random variables. Thus, `Y` lives in the vector space generated by `vectors` of Exponential distributions. This allows the user to decide the mean and covariance (by setting `loc` and `scale`), while preserving some properties of the Exponential distribution. In particular, the tails of `Y_i` will be (up to polynomial factors) exponentially decaying.

To see this last statement, note that the pdf of `Y_i` is the convolution of the pdf of `k` independent Exponential random variables. One can then show by induction that distributions with exponential (up to polynomial factors) tails are closed under convolution.

#### Examples

``````import tensorflow_probability as tfp
tfd = tfp.distributions

# Initialize a single 2-variate VectorExponential, supported on
# {(x, y) in R^2 : x > 0, y > 0}.

# The first component has pdf exp{-x}, the second 0.5 exp{-x / 2}
vex = tfd.VectorExponentialDiag(scale_diag=[1., 2.])

# Compute the pdf of an`R^2` observation; return a scalar.
vex.prob([3., 4.]).eval()  # shape: []

# Initialize a 2-batch of 3-variate Vector Exponential's.
loc = [[1., 2, 3],
[1., 0, 0]]              # shape: [2, 3]
scale_diag = [[1., 2, 3],
[0.5, 1, 1.5]]     # shape: [2, 3]

vex = tfd.VectorExponentialDiag(loc, scale_diag)

# Compute the pdf of two `R^3` observations; return a length-2 vector.
x = [[1.9, 2.2, 3.1],
[10., 1.0, 9.0]]     # shape: [2, 3]
vex.prob(x).eval()    # shape: [2]
``````

`loc` Floating-point `Tensor`. If this is set to `None`, `loc` is implicitly `0`. When specified, may have shape `[B1, ..., Bb, k]` where `b >= 0` and `k` is the event size.
`scale_diag` Non-zero, floating-point `Tensor` representing a diagonal matrix added to `scale`. May have shape `[B1, ..., Bb, k]`, `b >= 0`, and characterizes `b`-batches of `k x k` diagonal matrices added to `scale`. When both `scale_identity_multiplier` and `scale_diag` are `None` then `scale` is the `Identity`.
`scale_identity_multiplier` Non-zero, floating-point `Tensor` representing a scaled-identity-matrix added to `scale`. May have shape `[B1, ..., Bb]`, `b >= 0`, and characterizes `b`-batches of scaled `k x k` identity matrices added to `scale`. When both `scale_identity_multiplier` and `scale_diag` are `None` then `scale` is the `Identity`.
`validate_args` Python `bool`, default `False`. When `True` distribution parameters are checked for validity despite possibly degrading runtime performance. When `False` invalid inputs may silently render incorrect outputs.
`allow_nan_stats` Python `bool`, default `True`. When `True`, statistics (e.g., mean, mode, variance) use the value "`NaN`" to indicate the result is undefined. When `False`, an exception is raised if one or more of the statistic's batch members are undefined.
`name` Python `str` name prefixed to Ops created by this class.

`ValueError` if at most `scale_identity_multiplier` is specified.

`allow_nan_stats` Python `bool` describing behavior when a stat is undefined.

Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.

`batch_shape` Shape of a single sample from a single event index as a `TensorShape`.

May be partially defined or unknown.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

`bijector` Function transforming x => y.
`distribution` Base distribution, p(x).
`dtype` The `DType` of `Tensor`s handled by this `Distribution`.
`event_shape` Shape of a single sample from a single batch as a `TensorShape`.

May be partially defined or unknown.

`loc` The `loc` `Tensor` in `Y = scale @ X + loc`.
`name` Name prepended to all ops created by this `Distribution`.
`parameters` Dictionary of parameters used to instantiate this `Distribution`.
`reparameterization_type` Describes how samples from the distribution are reparameterized.

Currently this is one of the static instances `distributions.FULLY_REPARAMETERIZED` or `distributions.NOT_REPARAMETERIZED`.

`scale` The `scale` `LinearOperator` in `Y = scale @ X + loc`.
`validate_args` Python `bool` indicating possibly expensive checks are enabled.

## Methods

### `batch_shape_tensor`

View source

Shape of a single sample from a single event index as a 1-D `Tensor`.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

Args
`name` name to give to the op

Returns
`batch_shape` `Tensor`.

### `cdf`

View source

Cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

``````cdf(x) := P[X <= x]
``````

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.

Returns
`cdf` a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `copy`

View source

Creates a deep copy of the distribution.

Args
`**override_parameters_kwargs` String/value dictionary of initialization arguments to override with new values.

Returns
`distribution` A new instance of `type(self)` initialized from the union of self.parameters and override_parameters_kwargs, i.e., `dict(self.parameters, **override_parameters_kwargs)`.

### `covariance`

View source

Covariance.

Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-`k`, vector-valued distribution, it is calculated as,

``````Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]
``````

where `Cov` is a (batch of) `k x k` matrix, `0 <= (i, j) < k`, and `E` denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), `Covariance` shall return a (batch of) matrices under some vectorization of the events, i.e.,

``````Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]
``````

where `Cov` is a (batch of) `k' x k'` matrices, `0 <= (i, j) < k' = reduce_prod(event_shape)`, and `Vec` is some function mapping indices of this distribution's event dimensions to indices of a length-`k'` vector.

Args
`name` Python `str` prepended to names of ops created by this function.

Returns
`covariance` Floating-point `Tensor` with shape `[B1, ..., Bn, k', k']` where the first `n` dimensions are batch coordinates and `k' = reduce_prod(self.event_shape)`.

### `cross_entropy`

View source

Computes the (Shannon) cross entropy.

Denote this distribution (`self`) by `P` and the `other` distribution by `Q`. Assuming `P, Q` are absolutely continuous with respect to one another and permit densities `p(x) dr(x)` and `q(x) dr(x)`, (Shanon) cross entropy is defined as:

``````H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)
``````

where `F` denotes the support of the random variable `X ~ P`.

Args
`other` `tfp.distributions.Distribution` instance.
`name` Python `str` prepended to names of ops created by this function.

Returns
`cross_entropy` `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of (Shanon) cross entropy.

### `entropy`

View source

Shannon entropy in nats.

### `event_shape_tensor`

View source

Shape of a single sample from a single batch as a 1-D int32 `Tensor`.

Args
`name` name to give to the op

Returns
`event_shape` `Tensor`.

### `is_scalar_batch`

View source

Indicates that `batch_shape == []`.

Args
`name` Python `str` prepended to names of ops created by this function.

Returns
`is_scalar_batch` `bool` scalar `Tensor`.

### `is_scalar_event`

View source

Indicates that `event_shape == []`.

Args
`name` Python `str` prepended to names of ops created by this function.

Returns
`is_scalar_event` `bool` scalar `Tensor`.

### `kl_divergence`

View source

Computes the Kullback--Leibler divergence.

Denote this distribution (`self`) by `p` and the `other` distribution by `q`. Assuming `p, q` are absolutely continuous with respect to reference measure `r`, the KL divergence is defined as:

``````KL[p, q] = E_p[log(p(X)/q(X))]
= -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x)
= H[p, q] - H[p]
``````

where `F` denotes the support of the random variable `X ~ p`, `H[., .]` denotes (Shanon) cross entropy, and `H[.]` denotes (Shanon) entropy.

Args
`other` `tfp.distributions.Distribution` instance.
`name` Python `str` prepended to names of ops created by this function.