# tf.contrib.distributions.bijectors.RealNVP

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/distributions/python/ops/bijectors/real_nvp.py#L35-L238) |

RealNVP "affine coupling layer" for vector-valued events.

Inherits From: [`Bijector`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distributions/bijectors/Bijector)

    tf.contrib.distributions.bijectors.RealNVP(
        num_masked, shift_and_log_scale_fn, is_constant_jacobian=False,
        validate_args=False, name=None
    )

Real NVP models a normalizing flow on a `D`-dimensional distribution via a
single `D-d`-dimensional conditional distribution \[(Dinh et al., 2017)\]\[1\]:

`y[d:D] = y[d:D] * math_ops.exp(log_scale_fn(y[d:D])) + shift_fn(y[d:D])`
`y[0:d] = x[0:d]`

The last `D-d` units are scaled and shifted based on the first `d` units only,
while the first `d` units are 'masked' and left unchanged. Real NVP's
`shift_and_log_scale_fn` computes vector-valued quantities. For
scale-and-shift transforms that do not depend on any masked units, i.e.
`d=0`, use the `tfb.Affine` bijector with learned parameters instead.

Masking is currently only supported for base distributions with
`event_ndims=1`. For more sophisticated masking schemes like checkerboard or
channel-wise masking \[(Papamakarios et al., 2016)\[4\], use the `tfb.Permute`
bijector to re-order desired masked units into the first `d` units. For base
distributions with `event_ndims > 1`, use the `tfb.Reshape` bijector to
flatten the event shape.

Recall that the MAF bijector \[(Papamakarios et al., 2016)\]\[4\] implements a
normalizing flow via an autoregressive transformation. MAF and IAF have
opposite computational tradeoffs - MAF can train all units in parallel but
must sample units sequentially, while IAF must train units sequentially but
can sample in parallel. In contrast, Real NVP can compute both forward and
inverse computations in parallel. However, the lack of an autoregressive
transformations makes it less expressive on a per-bijector basis.

A "valid" `shift_and_log_scale_fn` must compute each `shift` (aka `loc` or
"mu" in \[Papamakarios et al. (2016)\]\[4\]) and `log(scale)` (aka "alpha" in
\[Papamakarios et al. (2016)\]\[4\]) such that each are broadcastable with the
arguments to `forward` and `inverse`, i.e., such that the calculations in
`forward`, `inverse` \[below\] are possible. For convenience,
`real_nvp_default_nvp` is offered as a possible `shift_and_log_scale_fn`
function.

NICE \[(Dinh et al., 2014)\]\[2\] is a special case of the Real NVP bijector
which discards the scale transformation, resulting in a constant-time
inverse-log-determinant-Jacobian. To use a NICE bijector instead of Real
NVP, `shift_and_log_scale_fn` should return `(shift, None)`, and
`is_constant_jacobian` should be set to `True` in the `RealNVP` constructor.
Calling `real_nvp_default_template` with `shift_only=True` returns one such
NICE-compatible `shift_and_log_scale_fn`.

Caching: the scalar input depth `D` of the base distribution is not known at
construction time. The first call to any of `forward(x)`, `inverse(x)`,
`inverse_log_det_jacobian(x)`, or `forward_log_det_jacobian(x)` memoizes
`D`, which is re-used in subsequent calls. This shape must be known prior to
graph execution (which is the case if using tf.layers).

#### Example Use

    import tensorflow_probability as tfp
    tfd = tfp.distributions
    tfb = tfp.bijectors

    # A common choice for a normalizing flow is to use a Gaussian for the base
    # distribution. (However, any continuous distribution would work.) E.g.,
    num_dims = 3
    num_samples = 1
    nvp = tfd.TransformedDistribution(
        distribution=tfd.MultivariateNormalDiag(loc=np.zeros(num_dims)),
        bijector=tfb.RealNVP(
            num_masked=2,
            shift_and_log_scale_fn=tfb.real_nvp_default_template(
                hidden_layers=[512, 512])))

    x = nvp.sample(num_samples)
    nvp.log_prob(x)
    nvp.log_prob(np.zeros([num_samples, num_dims]))

For more examples, see \[Jang (2018)\]\[3\].

#### References

\[1\]: Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density Estimation
using Real NVP. In *International Conference on Learning
Representations* , 2017. <https://arxiv.org/abs/1605.08803>

\[2\]: Laurent Dinh, David Krueger, and Yoshua Bengio. NICE: Non-linear
Independent Components Estimation. *arXiv preprint arXiv:1410.8516* ,
2014. <https://arxiv.org/abs/1410.8516>

\[3\]: Eric Jang. Normalizing Flows Tutorial, Part 2: Modern Normalizing Flows.
*Technical Report* , 2018. <http://blog.evjang.com/2018/01/nf2.html>

\[4\]: George Papamakarios, Theo Pavlakou, and Iain Murray. Masked
Autoregressive Flow for Density Estimation. In *Neural Information
Processing Systems* , 2017. <https://arxiv.org/abs/1705.07057>

<br />

<br />

<br />

| ## Args ||
|---|---|
| `num_masked` | Python `int` indicating that the first `d` units of the event should be masked. Must be in the closed interval `[1, D-1]`, where `D` is the event size of the base distribution. |
| `shift_and_log_scale_fn` | Python `callable` which computes `shift` and `log_scale` from both the forward domain (`x`) and the inverse domain (`y`). Calculation must respect the "autoregressive property" (see class docstring). Suggested default `masked_autoregressive_default_template(hidden_layers=...)`. Typically the function contains `tf.Variables` and is wrapped using [`tf.compat.v1.make_template`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/make_template). Returning `None` for either (both) `shift`, `log_scale` is equivalent to (but more efficient than) returning zero. |
| `is_constant_jacobian` | Python `bool`. Default: `False`. When `True` the implementation assumes `log_scale` does not depend on the forward domain (`x`) or inverse domain (`y`) values. (No validation is made; `is_constant_jacobian=False` is always safe but possibly computationally inefficient.) |
| `validate_args` | Python `bool` indicating whether arguments should be checked for correctness. |
| `name` | Python `str`, name given to ops managed by this object. |

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `ValueError` | If num_masked \< 1. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `dtype` | dtype of `Tensor`s transformable by this distribution. |
| `forward_min_event_ndims` | Returns the minimal number of dimensions bijector.forward operates on. |
| `graph_parents` | Returns this `Bijector`'s graph_parents as a Python list. |
| `inverse_min_event_ndims` | Returns the minimal number of dimensions bijector.inverse operates on. |
| `is_constant_jacobian` | Returns true iff the Jacobian matrix is not a function of x. <br /> > [!NOTE] > **Note:** Jacobian matrix is either constant for both forward and inverse or neither. |
| `name` | Returns the string name of this `Bijector`. |
| `validate_args` | Returns True if Tensor arguments will be validated. |

## Methods

### `forward`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L753-L768)

    forward(
        x, name='forward'
    )

Returns the forward `Bijector` evaluation, i.e., X = g(Y).

<br />

<br />

<br />

| Args ||
|---|---|
| `x` | `Tensor`. The input to the "forward" evaluation. |
| `name` | The name to give this op. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `Tensor`. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | if `self.dtype` is specified and `x.dtype` is not `self.dtype`. |
| `NotImplementedError` | if `_forward` is not implemented. |

<br />

### `forward_event_shape`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L677-L690)

    forward_event_shape(
        input_shape
    )

Shape of a single sample from a single batch as a `TensorShape`.

Same meaning as `forward_event_shape_tensor`. May be only partially defined.

<br />

<br />

<br />

| Args ||
|---|---|
| `input_shape` | `TensorShape` indicating event-portion shape passed into `forward` function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `forward_event_shape_tensor` | `TensorShape` indicating event-portion shape after applying `forward`. Possibly unknown. |

<br />

### `forward_event_shape_tensor`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L653-L670)

    forward_event_shape_tensor(
        input_shape, name='forward_event_shape_tensor'
    )

Shape of a single sample from a single batch as an `int32` 1D `Tensor`.

<br />

<br />

<br />

| Args ||
|---|---|
| `input_shape` | `Tensor`, `int32` vector indicating event-portion shape passed into `forward` function. |
| `name` | name to give to the op |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `forward_event_shape_tensor` | `Tensor`, `int32` vector indicating event-portion shape after applying `forward`. |

<br />

### `forward_log_det_jacobian`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L973-L997)

    forward_log_det_jacobian(
        x, event_ndims, name='forward_log_det_jacobian'
    )

Returns both the forward_log_det_jacobian.

<br />

<br />

<br />

| Args ||
|---|---|
| `x` | `Tensor`. The input to the "forward" Jacobian determinant evaluation. |
| `event_ndims` | Number of dimensions in the probabilistic events being transformed. Must be greater than or equal to `self.forward_min_event_ndims`. The result is summed over the final dimensions to produce a scalar Jacobian determinant for each event, i.e. it has shape `x.shape.ndims - event_ndims` dimensions. |
| `name` | The name to give this op. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `Tensor`, if this bijector is injective. If not injective this is not implemented. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | if `self.dtype` is specified and `y.dtype` is not `self.dtype`. |
| `NotImplementedError` | if neither `_forward_log_det_jacobian` nor {`_inverse`, `_inverse_log_det_jacobian`} are implemented, or this is a non-injective bijector. |

<br />

### `inverse`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L787-L804)

    inverse(
        y, name='inverse'
    )

Returns the inverse `Bijector` evaluation, i.e., X = g\^{-1}(Y).

<br />

<br />

<br />

| Args ||
|---|---|
| `y` | `Tensor`. The input to the "inverse" evaluation. |
| `name` | The name to give this op. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `Tensor`, if this bijector is injective. If not injective, returns the k-tuple containing the unique `k` points `(x1, ..., xk)` such that `g(xi) = y`. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | if `self.dtype` is specified and `y.dtype` is not `self.dtype`. |
| `NotImplementedError` | if `_inverse` is not implemented. |

<br />

### `inverse_event_shape`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L721-L734)

    inverse_event_shape(
        output_shape
    )

Shape of a single sample from a single batch as a `TensorShape`.

Same meaning as `inverse_event_shape_tensor`. May be only partially defined.

<br />

<br />

<br />

| Args ||
|---|---|
| `output_shape` | `TensorShape` indicating event-portion shape passed into `inverse` function. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `inverse_event_shape_tensor` | `TensorShape` indicating event-portion shape after applying `inverse`. Possibly unknown. |

<br />

### `inverse_event_shape_tensor`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L697-L714)

    inverse_event_shape_tensor(
        output_shape, name='inverse_event_shape_tensor'
    )

Shape of a single sample from a single batch as an `int32` 1D `Tensor`.

<br />

<br />

<br />

| Args ||
|---|---|
| `output_shape` | `Tensor`, `int32` vector indicating event-portion shape passed into `inverse` function. |
| `name` | name to give to the op |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `inverse_event_shape_tensor` | `Tensor`, `int32` vector indicating event-portion shape after applying `inverse`. |

<br />

### `inverse_log_det_jacobian`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/ops/distributions/bijector_impl.py#L871-L900)

    inverse_log_det_jacobian(
        y, event_ndims, name='inverse_log_det_jacobian'
    )

Returns the (log o det o Jacobian o inverse)(y).

Mathematically, returns: `log(det(dX/dY))(Y)`. (Recall that: `X=g^{-1}(Y)`.)

Note that `forward_log_det_jacobian` is the negative of this function,
evaluated at `g^{-1}(y)`.

<br />

<br />

<br />

| Args ||
|---|---|
| `y` | `Tensor`. The input to the "inverse" Jacobian determinant evaluation. |
| `event_ndims` | Number of dimensions in the probabilistic events being transformed. Must be greater than or equal to `self.inverse_min_event_ndims`. The result is summed over the final dimensions to produce a scalar Jacobian determinant for each event, i.e. it has shape `y.shape.ndims - event_ndims` dimensions. |
| `name` | The name to give this op. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| `Tensor`, if this bijector is injective. If not injective, returns the tuple of local log det Jacobians, `log(det(Dg_i^{-1}(y)))`, where `g_i` is the restriction of `g` to the `ith` partition `Di`. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | if `self.dtype` is specified and `y.dtype` is not `self.dtype`. |
| `NotImplementedError` | if `_inverse_log_det_jacobian` is not implemented. |

<br />