tfp.substrates.jax.distributions.Empirical

Empirical distribution.

Inherits From: Distribution

The Empirical distribution is parameterized by a [batch] multiset of samples. It describes the empirical measure (observations) of a variable.

Mathematical Details

The probability mass function (pmf) and cumulative distribution function (cdf) are

pmf(k; s1, ..., sn) = sum_i I(k)^{k == si} / n
I(k)^{k == si} == 1, if k == si, else 0.
cdf(k; s1, ..., sn) = sum_i I(k)^{k >= si} / n
I(k)^{k >= si} == 1, if k >= si, else 0.

Examples


# Initialize a empirical distribution with 4 scalar samples.
dist = Empirical(samples=[0., 1., 1., 2.])
dist.cdf(1.)
==> 0.75
dist.prob([0., 1.])
==> [0.25, 0.5] # samples will be broadcast to
                  [[0., 1., 1., 2.], [0., 1., 1., 2.]] to match event.

# Initialize a empirical distribution with a [2] batch of scalar samples.
dist = Empirical(samples=[[0., 1.], [1., 2.]])
dist.cdf([0., 2.])
==> [0.5, 1.]
dist.prob(0.)
==> [0.5, 0] # event will be broadcast to [0., 0.] to match samples.

# Initialize a empirical distribution with 4 vector-like samples.
dist = Empirical(samples=[[0., 0.], [0., 1.], [0., 1.], [1., 2.]],
                 event_ndims=1)
dist.cdf([0., 1.])
==> 0.75
dist.prob([[0., 1.], [1., 2.]])
==> [0.5, 0.25] # samples will be broadcast to shape [2, 4, 2] to match event.

# Initialize a empirical distribution with a [2] batch of vector samples.
dist = Empirical(samples=[[[0., 0.], [0., 1.]], [[0., 1.], [1., 2.]]],
                 event_ndims=1)
dist.cdf([[0., 0.], [0., 1.]])
==> [0.5, 0.5]
dist.prob([0., 1.])
==> [0.5, 1.] # event will be broadcast to shape [[0., 1.], [0., 1.]]
                to match samples.

samples Numeric Tensor of shape [B1, ..., Bk, S, E1, ..., En],k, n >= 0. Samples or batches of samples on which the distribution is based. The firstkdimensions index into a batch of independent distributions. Length ofSdimension determines number of samples in each multiset. The lastndimension represents samples for each distribution. n is specified by argument event_ndims. </td> </tr><tr> <td>event_ndims</td> <td> Pythonint32, default0. number of dimensions for each event. When0this distribution has scalar samples. When1this distribution has vector-like samples. </td> </tr><tr> <td>validate_args</td> <td> Pythonbool, defaultFalse. WhenTruedistribution parameters are checked for validity despite possibly degrading runtime performance. WhenFalseinvalid inputs may silently render incorrect outputs. </td> </tr><tr> <td>allow_nan_stats</td> <td> Pythonbool, defaultTrue. WhenTrue, statistics (e.g., mean, mode, variance) use the valueNaNto indicate the result is undefined. WhenFalse, an exception is raised if one or more of the statistic's batch members are undefined. </td> </tr><tr> <td>name</td> <td> Pythonstr` name prefixed to Ops created by this class.

ValueError if the rank of samples is statically known and less than event_ndims + 1.

allow_nan_stats Python bool describing behavior when a stat is undefined.

Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.

batch_shape Shape of a single sample from a single event index as a TensorShape.

May be partially defined or unknown.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

dtype The DType of Tensors handled by this Distribution.
event_shape Shape of a single sample from a single batch as a TensorShape.

May be partially defined or unknown.

name Name prepended to all ops created by this Distribution.
parameters Dictionary of parameters used to instantiate this Distribution.
reparameterization_type Describes how samples from the distribution are reparameterized.

Currently this is one of the static instances tfd.FULLY_REPARAMETERIZED or tfd.NOT_REPARAMETERIZED.

samples Distribution parameter.
trainable_variables

validate_args Python bool indicating possibly expensive checks are enabled.
variables

Methods

batch_shape_tensor

View source

Shape of a single sample from a single event index as a 1-D Tensor.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

Args
name name to give to the op

Returns
batch_shape Tensor.

cdf

View source

Cumulative distribution function.

Given random variable X, the cumulative distribution function cdf is:

cdf(x) := P[X <= x]

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
cdf a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

compute_num_samples

View source

Compute and return the number of values in self.samples.

Returns
num_samples int32 Tensor containing the number of entries in self.samples. If self.samples has shape [..., S, E1, ..., Ee] where the E's are event dims, this method returns a Tensor whose values is S.

copy

View source

Creates a deep copy of the distribution.

Args
**override_parameters_kwargs String/value dictionary of initialization arguments to override with new values.

Returns
distribution A new instance of type(self) initialized from the union of self.parameters and override_parameters_kwargs, i.e., dict(self.parameters, **override_parameters_kwargs).

covariance

View source

Covariance.

Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-k, vector-valued distribution, it is calculated as,

Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]

where Cov is a (batch of) k x k matrix, 0 <= (i, j) < k, and E denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), Covariance shall return a (batch of) matrices under some vectorization of the events, i.e.,

Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]

where Cov is a (batch of) k' x k' matrices, 0 <= (i, j) < k' = reduce_prod(event_shape), and Vec is some function mapping indices of this distribution's event dimensions to indices of a length-k' vector.

Args
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
covariance Floating-point Tensor with shape [B1, ..., Bn, k', k'] where the first n dimensions are batch coordinates and k' = reduce_prod(self.event_shape).

cross_entropy

View source

Computes the (Shannon) cross entropy.

Denote this distribution (self) by P and the other distribution by Q. Assuming P, Q are absolutely continuous with respect to one another and permit densities p(x) dr(x) and q(x) dr(x), (Shannon) cross entropy is defined as:

H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)

where F denotes the support of the random variable X ~ P.

Args
other tfp.distributions.Distribution instance.
name Python str prepended to names of ops created by this function.

Returns
cross_entropy self.dtype Tensor with shape [B1, ..., Bn] representing n different calculations of (Shannon) cross entropy.

entropy

View source

Shannon entropy in nats.

event_shape_tensor

View source

Shape of a single sample from a single batch as a 1-D int32 Tensor.

Args
name name to give to the op

Returns
event_shape Tensor.

is_scalar_batch

View source

Indicates that batch_shape == [].

Args
name Python str prepended to names of ops created by this function.

Returns
is_scalar_batch bool scalar Tensor.

is_scalar_event

View source

Indicates that event_shape == [].

Args
name Python str prepended to names of ops created by this function.

Returns
is_scalar_event bool scalar Tensor.

kl_divergence

View source

Computes the Kullback--Leibler divergence.

Denote this distribution (self) by p and the other distribution by q. Assuming p, q are absolutely continuous with respect to reference measure r, the KL divergence is defined as:

KL[p, q] = E_p[log(p(X)/q(X))]
         = -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x)
         = H[p, q] - H[p]

where F denotes the support of the random variable X ~ p, H[., .] denotes (Shannon) cross entropy, and H[.] denotes (Shannon) entropy.

Args
other tfp.distributions.Distribution instance.
name Python str prepended to names of ops created by this function.

Returns
kl_divergence self.dtype Tensor with shape [B1, ..., Bn] representing n different calculations of the Kullback-Leibler divergence.

log_cdf

View source

Log cumulative distribution function.

Given random variable X, the cumulative distribution function cdf is:

log_cdf(x) := Log[ P[X <= x] ]

Often, a numerical approximation can be used for log_cdf(x) that yields a more accurate answer than simply taking the logarithm of the cdf when x << -1.

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
logcdf a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

log_prob

View source

Log probability density/mass function.

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
log_prob a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

log_survival_function

View source

Log survival function.

Given random variable X, the survival function is defined:

log_survival_function(x) = Log[ P[X > x] ]
                         = Log[ 1 - P[X <= x] ]
                         = Log[ 1 - cdf(x) ]

Typically, different numerical approximations can be used for the log survival function, which are more accurate than 1 - cdf(x) when x >> 1.

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

mean

View source

Mean.

mode

View source

Mode.

param_shapes

View source

Shapes of parameters given the desired shape of a call to sample().

This is a class method that describes what key/value arguments are required to instantiate the given Distribution so that a particular shape is returned for that instance's call to sample().

Subclasses should override class method _param_shapes.

Args
sample_shape Tensor or python list/tuple. Desired shape of a call to sample().
name name to prepend ops with.

Returns
dict of parameter name to Tensor shapes.

param_static_shapes

View source

param_shapes with static (i.e. TensorShape) shapes.

This is a class method that describes what key/value arguments are required to instantiate the given Distribution so that a particular shape is returned for that instance's call to sample(). Assumes that the sample's shape is known statically.

Subclasses should override class method _param_shapes to return constant-valued tensors when constant values are fed.

Args
sample_shape TensorShape or python list/tuple. Desired shape of a call to sample().

Returns
dict of parameter name to TensorShape.

Raises
ValueError if sample_shape is a TensorShape and is not fully defined.

prob

View source

Probability density/mass function.

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
prob a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

quantile

View source

Quantile function. Aka 'inverse cdf' or 'percent point function'.

Given random variable X and p in [0, 1], the quantile is:

quantile(p) := x such that P[X <= x] == p

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
quantile a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

sample

View source

Generate samples of the specified shape.

Note that a call to sample() without arguments will generate a single sample.

Args
sample_shape 0D or 1D int32 Tensor. Shape of the generated samples.
seed Python integer or tfp.util.SeedStream instance, for seeding PRNG.
name name to give to the op.
**kwargs Named arguments forwarded to subclass implementation.

Returns
samples a Tensor with prepended dimensions sample_shape.

stddev

View source

Standard deviation.

Standard deviation is defined as,

stddev = E[(X - E[X])**2]**0.5

where X is the random variable associated with this distribution, E denotes expectation, and stddev.shape = batch_shape + event_shape.

Args
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
stddev Floating-point Tensor with shape identical to batch_shape + event_shape, i.e., the same shape as self.mean().

survival_function

View source

Survival function.

Given random variable X, the survival function is defined:

survival_function(x) = P[X > x]
                     = 1 - P[X <= x]
                     = 1 - cdf(x).

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

variance

View source

Variance.

Variance is defined as,

Var = E[(X - E[X])**2]

where X is the random variable associated with this distribution, E denotes expectation, and Var.shape = batch_shape + event_shape.

Args
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

Returns
variance Floating-point Tensor with shape identical to batch_shape + event_shape, i.e., the same shape as self.mean().

__getitem__

View source

Slices the batch axes of this distribution, returning a new instance.

b = tfd.Bernoulli(logits=tf.zeros([3, 5, 7, 9]))
b.batch_shape  # => [3, 5, 7, 9]
b2 = b[:, tf.newaxis, ..., -2:, 1::2]
b2.batch_shape  # => [3, 1, 5, 2, 4]

x = tf.random.stateless_normal([5, 3, 2, 2])
cov = tf.matmul(x, x, transpose_b=True)
chol = tf.linalg.cholesky(cov)
loc = tf.random.stateless_normal([4, 1, 3, 1])
mvn = tfd.MultivariateNormalTriL(loc, chol)
mvn.batch_shape  # => [4, 5, 3]
mvn.event_shape  # => [2]
mvn2 = mvn[:, 3:, ..., ::-1, tf.newaxis]
mvn2.batch_shape  # => [4, 2, 3, 1]
mvn2.event_shape  # => [2]

Args
slices slices from the [] operator

Returns
dist A new tfd.Distribution instance with sliced parameters.

__iter__

View source