View source on GitHub
|
Independent distribution from batch of distributions.
Inherits From: Distribution
tf.contrib.distributions.Independent(
distribution, reinterpreted_batch_ndims=None, validate_args=False, name=None
)
This distribution is useful for regarding a collection of independent,
non-identical distributions as a single random variable. For example, the
Independent distribution composed of a collection of Bernoulli
distributions might define a distribution over an image (where each
Bernoulli is a distribution over each pixel).
More precisely, a collection of B (independent) E-variate random variables
(rv) {X_1, ..., X_B}, can be regarded as a [B, E]-variate random variable
(X_1, ..., X_B) with probability
p(x_1, ..., x_B) = p_1(x_1) * ... * p_B(x_B) where p_b(X_b) is the
probability of the b-th rv. More generally B, E can be arbitrary shapes.
Similarly, the Independent distribution specifies a distribution over [B,
E]-shaped events. It operates by reinterpreting the rightmost batch dims as
part of the event dimensions. The reinterpreted_batch_ndims parameter
controls the number of batch dims which are absorbed as event dims;
reinterpreted_batch_ndims < len(batch_shape). For example, the log_prob
function entails a reduce_sum over the rightmost reinterpreted_batch_ndims
after calling the base distribution's log_prob. In other words, since the
batch dimension(s) index independent distributions, the resultant multivariate
will have independent components.
Mathematical Details
The probability function is,
prob(x; reinterpreted_batch_ndims) = tf.reduce_prod(
dist.prob(x),
axis=-1-range(reinterpreted_batch_ndims))
Examples
import tensorflow_probability as tfp
tfd = tfp.distributions
# Make independent distribution from a 2-batch Normal.
ind = tfd.Independent(
distribution=tfd.Normal(loc=[-1., 1], scale=[0.1, 0.5]),
reinterpreted_batch_ndims=1)
# All batch dims have been "absorbed" into event dims.
ind.batch_shape # ==> []
ind.event_shape # ==> [2]
# Make independent distribution from a 2-batch bivariate Normal.
ind = tfd.Independent(
distribution=tfd.MultivariateNormalDiag(
loc=[[-1., 1], [1, -1]],
scale_identity_multiplier=[1., 0.5]),
reinterpreted_batch_ndims=1)
# All batch dims have been "absorbed" into event dims.
ind.batch_shape # ==> []
ind.event_shape # ==> [2, 2]
Args | |
|---|---|
distribution
|
The base distribution instance to transform. Typically an
instance of Distribution.
|
reinterpreted_batch_ndims
|
Scalar, integer number of rightmost batch dims
which will be regarded as event dims. When None all but the first
batch axis (batch axis 0) will be transferred to event dimensions
(analogous to tf.compat.v1.layers.flatten).
|
validate_args
|
Python bool. Whether to validate input with asserts.
If validate_args is False, and the inputs are invalid,
correct behavior is not guaranteed.
|
name
|
The name for ops managed by the distribution.
Default value: Independent + distribution.name.
|
Raises | |
|---|---|
ValueError
|
if reinterpreted_batch_ndims exceeds
distribution.batch_ndims
|
Attributes | |
|---|---|
allow_nan_stats
|
Python bool describing behavior when a stat is undefined.
Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined. |
batch_shape
|
Shape of a single sample from a single event index as a TensorShape.
May be partially defined or unknown. The batch dimensions are indexes into independent, non-identical parameterizations of this distribution. |
distribution
|
|
dtype
|
The DType of Tensors handled by this Distribution.
|
event_shape
|
Shape of a single sample from a single batch as a TensorShape.
May be partially defined or unknown. |
name
|
Name prepended to all ops created by this Distribution.
|
parameters
|
Dictionary of parameters used to instantiate this Distribution.
|
reinterpreted_batch_ndims
|
|
reparameterization_type
|
Describes how samples from the distribution are reparameterized.
Currently this is one of the static instances
|
validate_args
|
Python bool indicating possibly expensive checks are enabled.
|
Methods
batch_shape_tensor
batch_shape_tensor(
name='batch_shape_tensor'
)
Shape of a single sample from a single event index as a 1-D Tensor.
The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.
| Args | |
|---|---|
name
|
name to give to the op |
| Returns | |
|---|---|
batch_shape
|
Tensor.
|
cdf
cdf(
value, name='cdf'
)
Cumulative distribution function.
Given random variable X, the cumulative distribution function cdf is:
cdf(x) := P[X <= x]
| Args | |
|---|---|
value
|
float or double Tensor.
|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
cdf
|
a Tensor of shape sample_shape(x) + self.batch_shape with
values of type self.dtype.
|
copy
copy(
**override_parameters_kwargs
)
Creates a deep copy of the distribution.
| Args | |
|---|---|
**override_parameters_kwargs
|
String/value dictionary of initialization arguments to override with new values. |
| Returns | |
|---|---|
distribution
|
A new instance of type(self) initialized from the union
of self.parameters and override_parameters_kwargs, i.e.,
dict(self.parameters, **override_parameters_kwargs).
|
covariance
covariance(
name='covariance'
)
Covariance.
Covariance is (possibly) defined only for non-scalar-event distributions.
For example, for a length-k, vector-valued distribution, it is calculated
as,
Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]
where Cov is a (batch of) k x k matrix, 0 <= (i, j) < k, and E
denotes expectation.
Alternatively, for non-vector, multivariate distributions (e.g.,
matrix-valued, Wishart), Covariance shall return a (batch of) matrices
under some vectorization of the events, i.e.,
Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]
where Cov is a (batch of) k' x k' matrices,
0 <= (i, j) < k' = reduce_prod(event_shape), and Vec is some function
mapping indices of this distribution's event dimensions to indices of a
length-k' vector.
| Args | |
|---|---|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
covariance
|
Floating-point Tensor with shape [B1, ..., Bn, k', k']
where the first n dimensions are batch coordinates and
k' = reduce_prod(self.event_shape).
|
cross_entropy
cross_entropy(
other, name='cross_entropy'
)
Computes the (Shannon) cross entropy.
Denote this distribution (self) by P and the other distribution by
Q. Assuming P, Q are absolutely continuous with respect to
one another and permit densities p(x) dr(x) and q(x) dr(x), (Shanon)
cross entropy is defined as:
H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)
where F denotes the support of the random variable X ~ P.
| Args | |
|---|---|
other
|
tfp.distributions.Distribution instance.
|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
cross_entropy
|
self.dtype Tensor with shape [B1, ..., Bn]
representing n different calculations of (Shanon) cross entropy.
|
entropy
entropy(
name='entropy'
)
Shannon entropy in nats.
event_shape_tensor
event_shape_tensor(
name='event_shape_tensor'
)
Shape of a single sample from a single batch as a 1-D int32 Tensor.
| Args | |
|---|---|
name
|
name to give to the op |
| Returns | |
|---|---|
event_shape
|
Tensor.
|
is_scalar_batch
is_scalar_batch(
name='is_scalar_batch'
)
Indicates that batch_shape == [].
| Args | |
|---|---|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
is_scalar_batch
|
bool scalar Tensor.
|
is_scalar_event
is_scalar_event(
name='is_scalar_event'
)
Indicates that event_shape == [].
| Args | |
|---|---|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
is_scalar_event
|
bool scalar Tensor.
|
kl_divergence
kl_divergence(
other, name='kl_divergence'
)
Computes the Kullback--Leibler divergence.
Denote this distribution (self) by p and the other distribution by
q. Assuming p, q are absolutely continuous with respect to reference
measure r, the KL divergence is defined as:
KL[p, q] = E_p[log(p(X)/q(X))]
= -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x)
= H[p, q] - H[p]
where F denotes the support of the random variable X ~ p, H[., .]
denotes (Shanon) cross entropy, and H[.] denotes (Shanon) entropy.
| Args | |
|---|---|
other
|
tfp.distributions.Distribution instance.
|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
kl_divergence
|
self.dtype Tensor with shape [B1, ..., Bn]
representing n different calculations of the Kullback-Leibler
divergence.
|
log_cdf
log_cdf(
value, name='log_cdf'
)
Log cumulative distribution function.
Given random variable X, the cumulative distribution function cdf is:
log_cdf(x) := Log[ P[X <= x] ]
Often, a numerical approximation can be used for log_cdf(x) that yields
a more accurate answer than simply taking the logarithm of the cdf when
x << -1.
| Args | |
|---|---|
value
|
float or double Tensor.
|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
logcdf
|
a Tensor of shape sample_shape(x) + self.batch_shape with
values of type self.dtype.
|
log_prob
log_prob(
value, name='log_prob'
)
Log probability density/mass function.
| Args | |
|---|---|
value
|
float or double Tensor.
|
name
|
Python str prepended to names of ops created by this function.
|
| Returns | |
|---|---|
log_prob
|
a Tensor of shape sample_shape(x) + self.batch_shape with
values of type |