# tf.contrib.distributions.Autoregressive

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/distributions/python/ops/autoregressive.py#L29-L222) |

Autoregressive distributions.

Inherits From: [`Distribution`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distributions/Distribution)

    tf.contrib.distributions.Autoregressive(
        distribution_fn, sample0=None, num_steps=None, validate_args=False,
        allow_nan_stats=True, name='Autoregressive'
    )

The Autoregressive distribution enables learning (often) richer multivariate
distributions by repeatedly applying a [diffeomorphic](https://en.wikipedia.org/wiki/Diffeomorphism) transformation (such as
implemented by `Bijector`s). Regarding terminology,

"Autoregressive models decompose the joint density as a product of
conditionals, and model each conditional in turn. Normalizing flows
transform a base density (e.g. a standard Gaussian) into the target density
by an invertible transformation with tractable Jacobian." \[(Papamakarios et
al., 2016)\]\[1\]

In other words, the "autoregressive property" is equivalent to the
decomposition, `p(x) = prod{ p(x[i] | x[0:i]) : i=0, ..., d }`. The provided
`shift_and_log_scale_fn`, `masked_autoregressive_default_template`, achieves
this property by zeroing out weights in its `masked_dense` layers.

Practically speaking the autoregressive property means that there exists a
permutation of the event coordinates such that each coordinate is a
diffeomorphic function of only preceding coordinates
\[(van den Oord et al., 2016)\]\[2\].

#### Mathematical Details

The probability function is

    prob(x; fn, n) = fn(x).prob(x)

And a sample is generated by

    x = fn(...fn(fn(x0).sample()).sample()).sample()

where the ellipses (`...`) represent `n-2` composed calls to `fn`, `fn`
constructs a [`tfp.distributions.Distribution`](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/Distribution)-like instance, and `x0` is a
fixed initializing `Tensor`.

#### Examples

    import tensorflow_probability as tfp
    tfd = tfp.distributions

    def normal_fn(self, event_size):
      n = event_size * (event_size + 1) / 2
      p = tf.Variable(tfd.Normal(loc=0., scale=1.).sample(n))
      affine = tfd.bijectors.Affine(
          scale_tril=tfd.fill_triangular(0.25 * p))
      def _fn(samples):
        scale = math_ops.exp(affine.forward(samples)).eval()
        return independent_lib.Independent(
            normal_lib.Normal(loc=0., scale=scale, validate_args=True),
            reinterpreted_batch_ndims=1)
      return _fn

    batch_and_event_shape = [3, 2, 4]
    sample0 = array_ops.zeros(batch_and_event_shape)
    ar = autoregressive_lib.Autoregressive(
        self._normal_fn(batch_and_event_shape[-1]), sample0)
    x = ar.sample([6, 5])
    # ==> x.shape = [6, 5, 3, 2, 4]
    prob_x = ar.prob(x)
    # ==> x.shape = [6, 5, 3, 2]

#### References

\[1\]: George Papamakarios, Theo Pavlakou, and Iain Murray. Masked
Autoregressive Flow for Density Estimation. In *Neural Information
Processing Systems* , 2017. <https://arxiv.org/abs/1705.07057>

\[2\]: Aaron van den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt,
Alex Graves, and Koray Kavukcuoglu. Conditional Image Generation with
PixelCNN Decoders. In *Neural Information Processing Systems* , 2016.
<https://arxiv.org/abs/1606.05328>

<br />

<br />

<br />

| ## Args ||
|---|---|
| `distribution_fn` | Python `callable` which constructs a [`tfp.distributions.Distribution`](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/Distribution)-like instance from a `Tensor` (e.g., `sample0`). The function must respect the "autoregressive property", i.e., there exists a permutation of event such that each coordinate is a diffeomorphic function of on preceding coordinates. |
| `sample0` | Initial input to `distribution_fn`; used to build the distribution in `__init__` which in turn specifies this distribution's properties, e.g., `event_shape`, `batch_shape`, `dtype`. If unspecified, then `distribution_fn` should be default constructable. |
| `num_steps` | Number of times `distribution_fn` is composed from samples, e.g., `num_steps=2` implies `distribution_fn(distribution_fn(sample0).sample(n)).sample()`. |
| `validate_args` | Python `bool`. Whether to validate input with asserts. If `validate_args` is `False`, and the inputs are invalid, correct behavior is not guaranteed. |
| `allow_nan_stats` | Python `bool`, default `True`. When `True`, statistics (e.g., mean, mode, variance) use the value "`NaN`" to indicate the result is undefined. When `False`, an exception is raised if one or more of the statistic's batch members are undefined. |
| `name` | Python `str` name prefixed to Ops created by this class. Default value: "Autoregressive". |

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `ValueError` | if `num_steps` and `distribution_fn(sample0).event_shape.num_elements()` are both `None`. |
| `ValueError` | if `num_steps < 1`. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `allow_nan_stats` | Python `bool` describing behavior when a stat is undefined. <br /> Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E\[(X - mean)\*\*2\] is also undefined. |
| `batch_shape` | Shape of a single sample from a single event index as a `TensorShape`. <br /> May be partially defined or unknown. The batch dimensions are indexes into independent, non-identical parameterizations of this distribution. |
| `distribution0` | <br /> <br /> |
| `distribution_fn` | <br /> <br /> |
| `dtype` | The `DType` of `Tensor`s handled by this `Distribution`. |
| `event_shape` | Shape of a single sample from a single batch as a `TensorShape`. <br /> May be partially defined or unknown. |
| `name` | Name prepended to all ops created by this `Distribution`. |
| `num_steps` | <br /> <br /> |
| `parameters` | Dictionary of parameters used to instantiate this `Distribution`. |
| `reparameterization_type` | Describes how samples from the distribution are reparameterized. <br /> Currently this is one of the static instances [`distributions.FULLY_REPARAMETERIZED`](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions#FULLY_REPARAMETERIZED) or [`distributions.NOT_REPARAMETERIZED`](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions#NOT_REPARAMETERIZED). |
| `sample0` | <br /> <br /> |
| `validate_args` | Python `bool` indicating possibly expensive checks are enabled. |

<br />

## Methods

### `batch_shape_tensor`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L637-L654)

    batch_shape_tensor(
        name='batch_shape_tensor'
    )

Shape of a single sample from a single event index as a 1-D `Tensor`.

The batch dimensions are indexes into independent, non-identical
parameterizations of this distribution.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | name to give to the op |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `batch_shape` | `Tensor`. |

<br />

### `cdf`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L881-L898)

    cdf(
        value, name='cdf'
    )

Cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

    cdf(x) := P[X <= x]

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `cdf` | a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. |

<br />

### `copy`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L615-L631)

    copy(
        **override_parameters_kwargs
    )

Creates a deep copy of the distribution.

> [!NOTE]
> **Note:** the copy distribution may continue to depend on the original initialization arguments.

<br />

<br />

<br />

| Args ||
|---|---|
| `**override_parameters_kwargs` | String/value dictionary of initialization arguments to override with new values. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `distribution` | A new instance of `type(self)` initialized from the union of self.parameters and override_parameters_kwargs, i.e., `dict(self.parameters, **override_parameters_kwargs)`. |

<br />

### `covariance`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1094-L1131)

    covariance(
        name='covariance'
    )

Covariance.

Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-`k`, vector-valued distribution, it is calculated
as,

    Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]

where `Cov` is a (batch of) `k x k` matrix, `0 <= (i, j) < k`, and `E`
denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g.,
matrix-valued, Wishart), `Covariance` shall return a (batch of) matrices
under some vectorization of the events, i.e.,

    Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]

where `Cov` is a (batch of) `k' x k'` matrices,
`0 <= (i, j) < k' = reduce_prod(event_shape)`, and `Vec` is some function
mapping indices of this distribution's event dimensions to indices of a
length-`k'` vector.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `covariance` | Floating-point `Tensor` with shape `[B1, ..., Bn, k', k']` where the first `n` dimensions are batch coordinates and `k' = reduce_prod(self.event_shape)`. |

<br />

### `cross_entropy`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1146-L1169)

    cross_entropy(
        other, name='cross_entropy'
    )

Computes the (Shannon) cross entropy.

Denote this distribution (`self`) by `P` and the `other` distribution by
`Q`. Assuming `P, Q` are absolutely continuous with respect to
one another and permit densities `p(x) dr(x)` and `q(x) dr(x)`, (Shanon)
cross entropy is defined as:

    H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)

where `F` denotes the support of the random variable `X ~ P`.

<br />

<br />

<br />

| Args ||
|---|---|
| `other` | [`tfp.distributions.Distribution`](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/Distribution) instance. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `cross_entropy` | `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of (Shanon) cross entropy. |

<br />

### `entropy`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L982-L985)

    entropy(
        name='entropy'
    )

Shannon entropy in nats.

### `event_shape_tensor`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L677-L691)

    event_shape_tensor(
        name='event_shape_tensor'
    )

Shape of a single sample from a single batch as a 1-D int32 `Tensor`.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | name to give to the op |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `event_shape` | `Tensor`. |

<br />

### `is_scalar_batch`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L721-L733)

    is_scalar_batch(
        name='is_scalar_batch'
    )

Indicates that `batch_shape == []`.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `is_scalar_batch` | `bool` scalar `Tensor`. |

<br />

### `is_scalar_event`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L707-L719)

    is_scalar_event(
        name='is_scalar_event'
    )

Indicates that `event_shape == []`.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `is_scalar_event` | `bool` scalar `Tensor`. |

<br />

### `kl_divergence`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1175-L1201)

    kl_divergence(
        other, name='kl_divergence'
    )

Computes the Kullback--Leibler divergence.

Denote this distribution (`self`) by `p` and the `other` distribution by
`q`. Assuming `p, q` are absolutely continuous with respect to reference
measure `r`, the KL divergence is defined as:

    KL[p, q] = E_p[log(p(X)/q(X))]
             = -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x)
             = H[p, q] - H[p]

where `F` denotes the support of the random variable `X ~ p`, `H[., .]`
denotes (Shanon) cross entropy, and `H[.]` denotes (Shanon) entropy.

<br />

<br />

<br />

| Args ||
|---|---|
| `other` | [`tfp.distributions.Distribution`](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/Distribution) instance. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `kl_divergence` | `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of the Kullback-Leibler divergence. |

<br />

### `log_cdf`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L842-L863)

    log_cdf(
        value, name='log_cdf'
    )

Log cumulative distribution function.

Given random variable `X`, the cumulative distribution function `cdf` is:

    log_cdf(x) := Log[ P[X <= x] ]

Often, a numerical approximation can be used for `log_cdf(x)` that yields
a more accurate answer than simply taking the logarithm of the `cdf` when
`x << -1`.

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `logcdf` | a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. |

<br />

### `log_prob`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L784-L795)

    log_prob(
        value, name='log_prob'
    )

Log probability density/mass function.

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `log_prob` | a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. |

<br />

### `log_survival_function`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L917-L939)

    log_survival_function(
        value, name='log_survival_function'
    )

Log survival function.

Given random variable `X`, the survival function is defined:

    log_survival_function(x) = Log[ P[X > x] ]
                             = Log[ 1 - P[X <= x] ]
                             = Log[ 1 - cdf(x) ]

Typically, different numerical approximations can be used for the log
survival function, which are more accurate than `1 - cdf(x)` when `x >> 1`.

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. ||

<br />

### `mean`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L991-L994)

    mean(
        name='mean'
    )

Mean.

### `mode`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1137-L1140)

    mode(
        name='mode'
    )

Mode.

### `param_shapes`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L497-L516)

    @classmethod
    param_shapes(
        sample_shape, name='DistributionParamShapes'
    )

Shapes of parameters given the desired shape of a call to `sample()`.

This is a class method that describes what key/value arguments are required
to instantiate the given `Distribution` so that a particular shape is
returned for that instance's call to `sample()`.

Subclasses should override class method `_param_shapes`.

<br />

<br />

<br />

| Args ||
|---|---|
| `sample_shape` | `Tensor` or python list/tuple. Desired shape of a call to `sample()`. |
| `name` | name to prepend ops with. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `dict` of parameter name to `Tensor` shapes. ||

<br />

### `param_static_shapes`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L518-L555)

    @classmethod
    param_static_shapes(
        sample_shape
    )

param_shapes with static (i.e. `TensorShape`) shapes.

This is a class method that describes what key/value arguments are required
to instantiate the given `Distribution` so that a particular shape is
returned for that instance's call to `sample()`. Assumes that the sample's
shape is known statically.

Subclasses should override class method `_param_shapes` to return
constant-valued tensors when constant values are fed.

<br />

<br />

<br />

| Args ||
|---|---|
| `sample_shape` | `TensorShape` or python list/tuple. Desired shape of a call to `sample()`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `dict` of parameter name to `TensorShape`. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | if `sample_shape` is a `TensorShape` and is not fully defined. |

<br />

### `prob`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L813-L824)

    prob(
        value, name='prob'
    )

Probability density/mass function.

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `prob` | a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. |

<br />

### `quantile`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1006-L1023)

    quantile(
        value, name='quantile'
    )

Quantile function. Aka "inverse cdf" or "percent point function".

Given random variable `X` and `p in [0, 1]`, the `quantile` is:

    quantile(p) := x such that P[X <= x] == p

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `quantile` | a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. |

<br />

### `sample`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L752-L766)

    sample(
        sample_shape=(), seed=None, name='sample'
    )

Generate samples of the specified shape.

Note that a call to `sample()` without arguments will generate a single
sample.

<br />

<br />

<br />

| Args ||
|---|---|
| `sample_shape` | 0D or 1D `int32` `Tensor`. Shape of the generated samples. |
| `seed` | Python integer seed for RNG |
| `name` | name to give to the op. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `samples` | a `Tensor` with prepended dimensions `sample_shape`. |

<br />

### `stddev`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1061-L1088)

    stddev(
        name='stddev'
    )

Standard deviation.

Standard deviation is defined as,

    stddev = E[(X - E[X])**2]**0.5

where `X` is the random variable associated with this distribution, `E`
denotes expectation, and `stddev.shape = batch_shape + event_shape`.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `stddev` | Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`. |

<br />

### `survival_function`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L957-L976)

    survival_function(
        value, name='survival_function'
    )

Survival function.

Given random variable `X`, the survival function is defined:

    survival_function(x) = P[X > x]
                         = 1 - P[X <= x]
                         = 1 - cdf(x).

<br />

<br />

<br />

| Args ||
|---|---|
| `value` | `float` or `double` `Tensor`. |
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. ||

<br />

### `variance`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/distribution.py#L1029-L1055)

    variance(
        name='variance'
    )

Variance.

Variance is defined as,

    Var = E[(X - E[X])**2]

where `X` is the random variable associated with this distribution, `E`
denotes expectation, and `Var.shape = batch_shape + event_shape`.

<br />

<br />

<br />

| Args ||
|---|---|
| `name` | Python `str` prepended to names of ops created by this function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `variance` | Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`. |

<br />