# tfp.distributions.QuantizedDistribution

Distribution representing the quantization `Y = ceiling(X)`.

Inherits From: `Distribution`

#### Definition in Terms of Sampling

``````
1. Draw X
2. Set Y <-- ceiling(X)
3. If Y < low, reset Y <-- low
4. If Y > high, reset Y <-- high
5. Return Y
``````

#### Definition in Terms of the Probability Mass Function

Given scalar random variable `X`, we define a discrete random variable `Y` supported on the integers as follows:

``````P[Y = j] := P[X <= low],  if j == low,
:= P[X > high - 1],  j == high,
:= 0, if j < low or j > high,
:= P[j - 1 < X <= j],  all other j.
``````

Conceptually, without cutoffs, the quantization process partitions the real line `R` into half open intervals, and identifies an integer `j` with the right endpoints:

``````R = ... (-2, -1](-1, 0](0, 1](1, 2](2, 3](3, 4] ...
j = ...      -1      0     1     2     3     4  ...
``````

`P[Y = j]` is the mass of `X` within the `jth` interval. If `low = 0`, and `high = 2`, then the intervals are redrawn and `j` is re-assigned:

``````R = (-infty, 0](0, 1](1, infty)
j =          0     1     2
``````

`P[Y = j]` is still the mass of `X` within the `jth` interval.

#### Examples

We illustrate a mixture of discretized logistic distributions [(Salimans et al., 2017)][1]. This is used, for example, for capturing 16-bit audio in WaveNet [(van den Oord et al., 2017)][2]. The values range in a 1-D integer domain of `[0, 2**16-1]`, and the discretization captures `P(x - 0.5 < X <= x + 0.5)` for all `x` in the domain excluding the endpoints. The lowest value has probability `P(X <= 0.5)` and the highest value has probability `P(2**16 - 1.5 < X)`.

Below we assume a `wavenet` function. It takes as `input` right-shifted audio samples of shape `[..., sequence_length]`. It returns a real-valued tensor of shape `[..., num_mixtures * 3]`, i.e., each mixture component has a `loc` and `scale` parameter belonging to the logistic distribution, and a `logits` parameter determining the unnormalized probability of that component.

``````tfd = tfp.distributions
tfb = tfp.bijectors

net = wavenet(inputs)
loc, unconstrained_scale, logits = tf.split(net,
num_or_size_splits=3,
axis=-1)
scale = tf.math.softplus(unconstrained_scale)

# Form mixture of discretized logistic distributions. Note we shift the
# logistic distribution by -0.5. This lets the quantization capture 'rounding'
# intervals, `(x-0.5, x+0.5]`, and not 'ceiling' intervals, `(x-1, x]`.
discretized_logistic_dist = tfd.QuantizedDistribution(
distribution=tfd.TransformedDistribution(
distribution=tfd.Logistic(loc=loc, scale=scale),
bijector=tfb.AffineScalar(shift=-0.5)),
low=0.,
high=2**16 - 1.)
mixture_dist = tfd.MixtureSameFamily(
mixture_distribution=tfd.Categorical(logits=logits),
components_distribution=discretized_logistic_dist)

neg_log_likelihood = -tf.reduce_sum(mixture_dist.log_prob(targets))
``````

After instantiating `mixture_dist`, we illustrate maximum likelihood by calculating its log-probability of audio samples as `target` and optimizing.

#### References

[1]: Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. PixelCNN++: Improving the PixelCNN with discretized logistic mixture likelihood and other modifications. International Conference on Learning Representations, 2017. https://arxiv.org/abs/1701.05517 [2]: Aaron van den Oord et al. Parallel WaveNet: Fast High-Fidelity Speech Synthesis. arXiv preprint arXiv:1711.10433, 2017. https://arxiv.org/abs/1711.10433

`distribution` The base distribution class to transform. Typically an instance of `Distribution`.
`low` `Tensor` with same `dtype` as this distribution and shape that broadcasts to that of samples but does not result in additional batch dimensions after broadcasting. Should be a whole number. Default `None`. If provided, base distribution's `prob` should be defined at `low`.
`high` `Tensor` with same `dtype` as this distribution and shape that broadcasts to that of samples but does not result in additional batch dimensions after broadcasting. Should be a whole number. Default `None`. If provided, base distribution's `prob` should be defined at `high - 1`. `high` must be strictly greater than `low`.
`validate_args` Python `bool`, default `False`. When `True` distribution parameters are checked for validity despite possibly degrading runtime performance. When `False` invalid inputs may silently render incorrect outputs.
`name` Python `str` name prefixed to Ops created by this class.

`TypeError` If `dist_cls` is not a subclass of `Distribution` or continuous.
`NotImplementedError` If the base distribution does not implement `cdf`.

`allow_nan_stats` Python `bool` describing behavior when a stat is undefined.

Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.

`batch_shape` Shape of a single sample from a single event index as a `TensorShape`.

May be partially defined or unknown.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

`distribution` Base distribution, p(x).
`dtype` The `DType` of `Tensor`s handled by this `Distribution`.
`event_shape` Shape of a single sample from a single batch as a `TensorShape`.

May be partially defined or unknown.

`high` Highest value that quantization returns.
`low` Lowest value that quantization returns.
`name` Name prepended to all ops created by this `Distribution`.
`name_scope` Returns a `tf.name_scope` instance for this class.
`parameters` Dictionary of parameters used to instantiate this `Distribution`.
`reparameterization_type` Describes how samples from the distribution are reparameterized.

Currently this is one of the static instances `tfd.FULLY_REPARAMETERIZED` or `tfd.NOT_REPARAMETERIZED`.

`submodules` Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

````a = tf.Module()`
`b = tf.Module()`
`c = tf.Module()`
`a.b = b`
`b.c = c`
`list(a.submodules) == [b, c]`
`True`
`list(b.submodules) == [c]`
`True`
`list(c.submodules) == []`
`True`
```

`trainable_variables` Sequence of trainable variables owned by this module and its submodules.

`validate_args` Python `bool` indicating possibly expensive checks are enabled.
`variables` Sequence of variables owned by this module and its submodules.

## Methods

### `batch_shape_tensor`

View source

Shape of a single sample from a single event index as a 1-D `Tensor`.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

Args
`name` name to give to the op

Returns
`batch_shape` `Tensor`.

### `cdf`

View source

Cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

``````cdf(x) := P[X <= x]
``````

Additional documentation from `QuantizedDistribution`:

For whole numbers `y`,

``````cdf(y) := P[Y <= y]
= 1, if y >= high,
= 0, if y < low,
= P[X <= y], otherwise.
``````

Since `Y` only has mass at whole numbers, `P[Y <= y] = P[Y <= floor(y)]`. This dictates that fractional `y` are first floored to a whole number, and then above definition applies.

The base distribution's `cdf` method must be defined on `y - 1`.

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`cdf` a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `copy`

View source

Creates a deep copy of the distribution.

Args
`**override_parameters_kwargs` String/value dictionary of initialization arguments to override with new values.

Returns
`distribution` A new instance of `type(self)` initialized from the union of self.parameters and override_parameters_kwargs, i.e., `dict(self.parameters, **override_parameters_kwargs)`.

### `covariance`

View source

Covariance.

Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-`k`, vector-valued distribution, it is calculated as,

``````Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]
``````

where `Cov` is a (batch of) `k x k` matrix, `0 <= (i, j) < k`, and `E` denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), `Covariance` shall return a (batch of) matrices under some vectorization of the events, i.e.,

``````Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]
``````

where `Cov` is a (batch of) `k' x k'` matrices, `0 <= (i, j) < k' = reduce_prod(event_shape)`, and `Vec` is some function mapping indices of this distribution's event dimensions to indices of a length-`k'` vector.

Args
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`covariance` Floating-point `Tensor` with shape `[B1, ..., Bn, k', k']` where the first `n` dimensions are batch coordinates and `k' = reduce_prod(self.event_shape)`.

### `cross_entropy`

View source

Computes the (Shannon) cross entropy.

Denote this distribution (`self`) by `P` and the `other` distribution by `Q`. Assuming `P, Q` are absolutely continuous with respect to one another and permit densities `p(x) dr(x)` and `q(x) dr(x)`, (Shannon) cross entropy is defined as:

``````H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)
``````

where `F` denotes the support of the random variable `X ~ P`.

Args
`other` `tfp.distributions.Distribution` instance.
`name` Python `str` prepended to names of ops created by this function.

Returns
`cross_entropy` `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of (Shannon) cross entropy.

### `entropy`

View source

Shannon entropy in nats.

### `event_shape_tensor`

View source

Shape of a single sample from a single batch as a 1-D int32 `Tensor`.

Args
`name` name to give to the op

Returns
`event_shape` `Tensor`.

### `is_scalar_batch`

View source

Indicates that `batch_shape == []`.

Args
`name` Python `str` prepended to names of ops created by this function.

Returns
`is_scalar_batch` `bool` scalar `Tensor`.

### `is_scalar_event`

View source

Indicates that `event_shape == []`.

Args
`name` Python `str` prepended to names of ops created by this function.

Returns
`is_scalar_event` `bool` scalar `Tensor`.

### `kl_divergence`

View source

Computes the Kullback--Leibler divergence.

Denote this distribution (`self`) by `p` and the `other` distribution by `q`. Assuming `p, q` are absolutely continuous with respect to reference measure `r`, the KL divergence is defined as:

``````KL[p, q] = E_p[log(p(X)/q(X))]
= -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x)
= H[p, q] - H[p]
``````

where `F` denotes the support of the random variable `X ~ p`, `H[., .]` denotes (Shannon) cross entropy, and `H[.]` denotes (Shannon) entropy.

Args
`other` `tfp.distributions.Distribution` instance.
`name` Python `str` prepended to names of ops created by this function.

Returns
`kl_divergence` `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of the Kullback-Leibler divergence.

### `log_cdf`

View source

Log cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

``````log_cdf(x) := Log[ P[X <= x] ]
``````

Often, a numerical approximation can be used for `log_cdf(x)` that yields a more accurate answer than simply taking the logarithm of the `cdf` when `x << -1`.

Additional documentation from `QuantizedDistribution`:

For whole numbers `y`,

``````cdf(y) := P[Y <= y]
= 1, if y >= high,
= 0, if y < low,
= P[X <= y], otherwise.
``````

Since `Y` only has mass at whole numbers, `P[Y <= y] = P[Y <= floor(y)]`. This dictates that fractional `y` are first floored to a whole number, and then above definition applies.

The base distribution's `log_cdf` method must be defined on `y - 1`.

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`logcdf` a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `log_prob`

View source

Log probability density/mass function.

Additional documentation from `QuantizedDistribution`:

For whole numbers `y`,

``````P[Y = y] := P[X <= low],  if y == low,
:= P[X > high - 1],  y == high,
:= 0, if j < low or y > high,
:= P[y - 1 < X <= y],  all other y.
``````

The base distribution's `log_cdf` method must be defined on `y - 1`. If the base distribution has a `log_survival_function` method results will be more accurate for large values of `y`, and in this case the `log_survival_function` must also be defined on `y - 1`.

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`log_prob` a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `log_survival_function`

View source

Log survival function.

Given random variable `X`, the survival function is defined:

``````log_survival_function(x) = Log[ P[X > x] ]
= Log[ 1 - P[X <= x] ]
= Log[ 1 - cdf(x) ]
``````

Typically, different numerical approximations can be used for the log survival function, which are more accurate than `1 - cdf(x)` when `x >> 1`.

Additional documentation from `QuantizedDistribution`:

For whole numbers `y`,

``````survival_function(y) := P[Y > y]
= 0, if y >= high,
= 1, if y < low,
= P[X <= y], otherwise.
``````

Since `Y` only has mass at whole numbers, `P[Y <= y] = P[Y <= floor(y)]`. This dictates that fractional `y` are first floored to a whole number, and then above definition applies.

The base distribution's `log_cdf` method must be defined on `y - 1`.

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

View source

Mean.

View source

Mode.

### `param_shapes`

View source

Shapes of parameters given the desired shape of a call to `sample()`.

This is a class method that describes what key/value arguments are required to instantiate the given `Distribution` so that a particular shape is returned for that instance's call to `sample()`.

Subclasses should override class method `_param_shapes`.

Args
`sample_shape` `Tensor` or python list/tuple. Desired shape of a call to `sample()`.
`name` name to prepend ops with.

Returns
`dict` of parameter name to `Tensor` shapes.

### `param_static_shapes`

View source

param_shapes with static (i.e. `TensorShape`) shapes.

This is a class method that describes what key/value arguments are required to instantiate the given `Distribution` so that a particular shape is returned for that instance's call to `sample()`. Assumes that the sample's shape is known statically.

Subclasses should override class method `_param_shapes` to return constant-valued tensors when constant values are fed.

Args
`sample_shape` `TensorShape` or python list/tuple. Desired shape of a call to `sample()`.

Returns
`dict` of parameter name to `TensorShape`.

Raises
`ValueError` if `sample_shape` is a `TensorShape` and is not fully defined.

### `prob`

View source

Probability density/mass function.

Additional documentation from `QuantizedDistribution`:

For whole numbers `y`,

``````P[Y = y] := P[X <= low],  if y == low,
:= P[X > high - 1],  y == high,
:= 0, if j < low or y > high,
:= P[y - 1 < X <= y],  all other y.
``````

The base distribution's `cdf` method must be defined on `y - 1`. If the base distribution has a `survival_function` method, results will be more accurate for large values of `y`, and in this case the `survival_function` must also be defined on `y - 1`.

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`prob` a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `quantile`

View source

Quantile function. Aka 'inverse cdf' or 'percent point function'.

Given random variable `X` and `p in [0, 1]`, the `quantile` is:

``````quantile(p) := x such that P[X <= x] == p
``````

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`quantile` a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `sample`

View source

Generate samples of the specified shape.

Note that a call to `sample()` without arguments will generate a single sample.

Args
`sample_shape` 0D or 1D `int32` `Tensor`. Shape of the generated samples.
`seed` Python integer or `tfp.util.SeedStream` instance, for seeding PRNG.
`name` name to give to the op.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`samples` a `Tensor` with prepended dimensions `sample_shape`.

### `stddev`

View source

Standard deviation.

Standard deviation is defined as,

``````stddev = E[(X - E[X])**2]**0.5
``````

where `X` is the random variable associated with this distribution, `E` denotes expectation, and `stddev.shape = batch_shape + event_shape`.

Args
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`stddev` Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`.

### `survival_function`

View source

Survival function.

Given random variable `X`, the survival function is defined:

``````survival_function(x) = P[X > x]
= 1 - P[X <= x]
= 1 - cdf(x).
``````

Additional documentation from `QuantizedDistribution`:

For whole numbers `y`,

``````survival_function(y) := P[Y > y]
= 0, if y >= high,
= 1, if y < low,
= P[X <= y], otherwise.
``````

Since `Y` only has mass at whole numbers, `P[Y <= y] = P[Y <= floor(y)]`. This dictates that fractional `y` are first floored to a whole number, and then above definition applies.

The base distribution's `cdf` method must be defined on `y - 1`.

Args
`value` `float` or `double` `Tensor`.
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`.

### `variance`

View source

Variance.

Variance is defined as,

``````Var = E[(X - E[X])**2]
``````

where `X` is the random variable associated with this distribution, `E` denotes expectation, and `Var.shape = batch_shape + event_shape`.

Args
`name` Python `str` prepended to names of ops created by this function.
`**kwargs` Named arguments forwarded to subclass implementation.

Returns
`variance` Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`.

### `with_name_scope`

Decorator to automatically enter the module name scope.

````class MyModule(tf.Module):`
`  @tf.Module.with_name_scope`
`  def __call__(self, x):`
`    if not hasattr(self, 'w'):`
`      self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))`
`    return tf.matmul(x, self.w)`
```

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

````mod = MyModule()`
`mod(tf.ones([1, 2]))`
`<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>`
`mod.w`
`<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,`
`numpy=..., dtype=float32)>`
```

Args
`method` The method to wrap.

Returns
The original method wrapped such that it enters the module's name scope.

### `__getitem__`

View source

Slices the batch axes of this distribution, returning a new instance.

``````b = tfd.Bernoulli(logits=tf.zeros([3, 5, 7, 9]))
b.batch_shape  # => [3, 5, 7, 9]
b2 = b[:, tf.newaxis, ..., -2:, 1::2]
b2.batch_shape  # => [3, 1, 5, 2, 4]

x = tf.random.normal([5, 3, 2, 2])
cov = tf.matmul(x, x, transpose_b=True)
chol = tf.cholesky(cov)
loc = tf.random.normal([4, 1, 3, 1])
mvn = tfd.MultivariateNormalTriL(loc, chol)
mvn.batch_shape  # => [4, 5, 3]
mvn.event_shape  # => [2]
mvn2 = mvn[:, 3:, ..., ::-1, tf.newaxis]
mvn2.batch_shape  # => [4, 2, 3, 1]
mvn2.event_shape  # => [2]
``````

Args
`slices` slices from the [] operator

Returns
`dist` A new `tfd.Distribution` instance with sliced parameters.

View source