tfp.substrates.numpy.bijectors.SoftClip

Bijector that approximates clipping as a continuous, differentiable map.

Inherits From: AutoCompositeTensorBijector, Bijector

View aliases

Main aliases

tfp.experimental.substrates.numpy.bijectors.SoftClip

tfp.substrates.numpy.bijectors.SoftClip(
    low=None,
    high=None,
    hinge_softness=None,
    validate_args=False,
    name='soft_clip'
)

The forward method takes unconstrained scalar x to a value y in [low, high]. For values within the interval and far from the bounds (low << x << high), this mapping is approximately the identity mapping.

b = tfb.SoftClip(low=-10., high=10.)
b.forward([-15., -7., 1., 9., 20.])
  # => [-9.993284, -6.951412,  0.9998932,  8.686738,  9.999954 ]

The softness of the clipping can be adjusted via the hinge_softness parameter. A sharp constraint (hinge_softness < 1.0) will approximate the identity mapping very well across almost all of its range, but may be numerically ill-conditioned at the boundaries. A soft constraint (hinge_softness > 1.0) corresponds to a smoother, better-conditioned mapping, but creates a larger distortion of its inputs.

b_hard = SoftClip(low=-5, high=5., hinge_softness=0.1)
b_soft.forward([-15., -7., 1., 9., 20.])
  # => [-10., -7., 1., 8.999995,  10.]

b_soft = SoftClip(low=-5, high=5., hinge_softness=10.0)
b_soft.forward([-15., -7., 1., 9., 20.])
  # => [-6.1985435, -3.369276,  0.16719627,  3.6655345,  7.1750355]

Note that the outputs are always in the interval [low, high], regardless of the hinge_softness.

Example use

A trivial application of this bijector is to constrain the values sampled from a distribution:

dist = tfd.TransformedDistribution(
  distribution=tfd.Normal(loc=0., scale=1.),
  bijector=tfb.SoftClip(low=-5., high=5.))
samples = dist.sample(100)  # => samples guaranteed in [-10., 10.]

A more useful application is to constrain the values considered during inference, preventing an inference algorithm from proposing values that cause numerical issues. For example, this model will return a log_prob of NaN when z is outside of the range [-5., 5.]:

dist = tfd.JointDistributionNamed({
  'z': tfd.Normal(0., 1.0)
  'x': lambda z: tfd.Normal(
                   loc=tf.log(25 - z**2), # Breaks if z >= 5 or z <= -5.
                   scale=1.)})

Using SoftClip allows us to keep an inference algorithm in the feasible region without distorting the inference geometry by very much:

target_log_prob_fn = lambda z: dist.log_prob(z=z, x=3.)  # Condition on x==3.

# Use SoftClip to ensure sampler stays within the numerically valid region.
mcmc_samples = tfp.mcmc.sample_chain(
  kernel=tfp.mcmc.TransformedTransitionKernel(
    tfp.mcmc.HamiltonianMonteCarlo(
      target_log_prob_fn=target_log_prob_fn,
      num_leapfrog_steps=2,
      step_size=0.1),
    bijector=tfb.SoftClip(-5., 5.)),
  trace_fn=None,
  current_state=0.,
  num_results=100)

Mathematical Details

The constraint is built by using softplus(x) = log(1 + exp(x)) as a smooth approximation to max(x, 0). In combination with affine transformations, this can implement a constraint to any scalar interval.

In particular, translating softplus gives a generic lower bound constraint:

max(x, low) =  max(x - low, 0) + low
            ~= softplus(x - low) + low
            := softlower(x)

Note that this quantity is always greater than low because softplus is positive-valued. We can also implement a soft upper bound:

min(x, high) =  min(x - high, 0) + high
             = -max(high - x, 0) + high
            ~= -softplus(high - x) + high
            := softupper(x)

which, similarly, is always less than high.

Composing these bounds as softupper(softlower(x)) gives a quantity bounded above by high, and bounded below by softupper(low) (because softupper is monotonic and its input is bounded below by low). In general, we will have softupper(low) < low, so we need to shrink the interval slightly (by (high - low) / (high - softupper(low))) to preserve the lower bound. The two-sided constraint is therefore:

softclip(x) := (softupper(softlower(x)) - high) *
                 (high - low) / (high - softupper(low)) + high
             = -softplus(high - low - softplus(x - low)) *
                 (high - low) / (softplus(high-low)) + high

Due to this rescaling, the bijector can be mildly asymmetric. Values of equal distance from the endpoints are mapped to values with slightly unequal distance from the endpoints; for example,

b = SoftConstrain(-1., 1.)
b.forward([-0.5., 0.5.])
  # => [-0.2527727 ,  0.19739306]

The degree of the asymmetry is proportional to the size of the rescaling correction, i.e., the extent to which softupper fails to be the identity map at the lower end of the interval. This is maximized when the upper and lower bounds are very close together relative to the hinge softness, as in the example above. Conversely, when the interval is wide, the required correction and asymmetry are very small.

Args
`low`	Optional float `Tensor` lower bound. If `None`, the lower-bound constraint is omitted. Default value: `None`.
`high`	Optional float `Tensor` upper bound. If `None`, the upper-bound constraint is omitted. Default value: `None`.
`hinge_softness`	Optional nonzero float `Tensor`. Controls the softness of the constraint at the boundaries; values outside of the constraint set are mapped into intervals of width approximately `log(2) * hinge_softness` on the interior of each boundary. High softness reserves more space for values outside of the constraint set, leading to greater distortion of inputs within the constraint set, but improved numerical stability near the boundaries. Default value: `None` (`1.0`).
`validate_args`	Python `bool` indicating whether arguments should be checked for correctness.
`name`	Python `str` name given to ops managed by this object.

Attributes
`dtype`
`forward_min_event_ndims`	Returns the minimal number of dimensions bijector.forward operates on. Multipart bijectors return structured `ndims`, which indicates the expected structure of their inputs. Some multipart bijectors, notably Composites, may return structures of `None`.
`graph_parents`	Returns this `Bijector`'s graph_parents as a Python list.
`has_static_min_event_ndims`	Returns True if the bijector has statically-known `min_event_ndims`.
`high`
`hinge_softness`
`inverse_min_event_ndims`	Returns the minimal number of dimensions bijector.inverse operates on. Multipart bijectors return structured `event_ndims`, which indicates the expected structure of their outputs. Some multipart bijectors, notably Composites, may return structures of `None`.
`is_constant_jacobian`	Returns true iff the Jacobian matrix is not a function of x. Note: Jacobian matrix is either constant for both forward and inverse or neither.
`low`
`name`	Returns the string name of this `Bijector`.
`parameters`	Dictionary of parameters used to instantiate this `Bijector`.
`trainable_variables`
`validate_args`	Returns True if Tensor arguments will be validated.
`variables`

Args
`x_event_ndims`	Optional Python `int` (structure) number of dimensions in a probabilistic event passed to `forward`; this must be greater than or equal to `self.forward_min_event_ndims`. If `None`, defaults to `self.forward_min_event_ndims`. Mutually exclusive with `y_event_ndims`. Default value: `None`.
`y_event_ndims`	Optional Python `int` (structure) number of dimensions in a probabilistic event passed to `inverse`; this must be greater than or equal to `self.inverse_min_event_ndims`. Mutually exclusive with `x_event_ndims`. Default value: `None`.

Args
`x`	`Tensor` (structure). The point at which to calculate the density.
`tangent_space`	`TangentSpace` or one of its subclasses. The tangent to the support manifold at `x`.
`backward_compat`	`bool` specifying whether to assume that the Bijector is dimension-preserving.
`**kwargs`	Optional keyword arguments forwarded to tangent space methods.

Args
`x`	`Tensor` (structure). The input to the 'forward' evaluation.
`name`	The name to give this op.
`**kwargs`	Named arguments forwarded to subclass implementation.

Raises
`TypeError`	if `self.dtype` is specified and `x.dtype` is not `self.dtype`.
`NotImplementedError`	if `_forward` is not implemented.

Args
`event_ndims`	Structure of Python and/or Tensor `int`s, and/or `None` values. The structure should match that of `self.forward_min_event_ndims`, and all non-`None` values must be greater than or equal to the corresponding value in `self.forward_min_event_ndims`.
`**kwargs`	Optional keyword arguments forwarded to nested bijectors.

Args
`input_shape`	`Tensor`, `int32` vector (structure) indicating event-portion shape passed into `forward` function.
`name`	name to give to the op

Args
`x`	`Tensor` (structure). The input to the 'forward' Jacobian determinant evaluation.
`event_ndims`	Optional number of dimensions in the probabilistic events being transformed; this must be greater than or equal to `self.forward_min_event_ndims`. If `event_ndims` is specified, the log Jacobian determinant is summed to produce a scalar log-determinant for each event. Otherwise (if `event_ndims` is `None`), no reduction is performed. Multipart bijectors require structured event_ndims, such that the batch rank `rank(y[i]) - event_ndims[i]` is the same for all elements `i` of the structured input. In most cases (with the exception of `tfb.JointMap`) they further require that `event_ndims[i] - self.inverse_min_event_ndims[i]` is the same for all elements `i` of the structured input. Default value: `None` (equivalent to `self.forward_min_event_ndims`).
`name`	The name to give this op.
`**kwargs`	Named arguments forwarded to subclass implementation.

Raises
`TypeError`	if `y`'s dtype is incompatible with the expected output dtype.
`NotImplementedError`	if neither `_forward_log_det_jacobian` nor {`_inverse`, `_inverse_log_det_jacobian`} are implemented, or this is a non-injective bijector.
`ValueError`	if the value of `event_ndims` is not valid for this bijector.

Raises
`TypeError`	if `y`'s structured dtype is incompatible with the expected output dtype.
`NotImplementedError`	if `_inverse` is not implemented.

Args
`output_shape`	`Tensor`, `int32` vector (structure) indicating event-portion shape passed into `inverse` function.
`name`	name to give to the op

Raises
`TypeError`	if `x`'s dtype is incompatible with the expected inverse-dtype.
`NotImplementedError`	if `_inverse_log_det_jacobian` is not implemented.
`ValueError`	if the value of `event_ndims` is not valid for this bijector.

Args
`value`	A `tfd.Distribution`, `tfb.Bijector`, or a (structure of) `Tensor`.
`name`	Python `str` name given to ops created by this function.
`**kwargs`	Additional keyword arguments passed into the created `tfd.TransformedDistribution`, `tfb.Bijector`, or `self.forward`.

tfp.substrates.numpy.bijectors.SoftClip

View aliases

Example use

Mathematical Details

Args

Attributes

Methods

copy

experimental_batch_shape

experimental_batch_shape_tensor

experimental_compute_density_correction

forward

forward_dtype

forward_event_ndims

forward_event_shape

forward_event_shape_tensor

forward_log_det_jacobian

inverse

inverse_dtype

inverse_event_ndims

inverse_event_shape

inverse_event_shape_tensor

inverse_log_det_jacobian

parameter_properties

__call__

Examples

__eq__

__getitem__

`copy`

`experimental_batch_shape`

`experimental_batch_shape_tensor`

`experimental_compute_density_correction`

`forward`

`forward_dtype`

`forward_event_ndims`

`forward_event_shape`

`forward_event_shape_tensor`

`forward_log_det_jacobian`

`inverse`

`inverse_dtype`

`inverse_event_ndims`

`inverse_event_shape`

`inverse_event_shape_tensor`

`inverse_log_det_jacobian`

`parameter_properties`

`call`

`eq`

`getitem`