Attend the Women in ML Symposium on December 7 Register now


Stay organized with collections Save and categorize content based on your preferences.

The Generalized Pareto distribution.

Inherits From: Distribution, AutoCompositeTensor

The Generalized Pareto distributions are a family of continuous distributions on the reals. Special cases include Exponential (when loc = 0, concentration = 0), Pareto (when concentration > 0, loc = scale / concentration), and Uniform (when concentration = -1).

This distribution is often used to model the tails of other distributions.

As a member of the location-scale family, X ~ GeneralizedPareto(loc=loc, scale=scale, concentration=conc) maps to Y ~ GeneralizedPareto(loc=0, scale=1, concentration=conc) via Y = (X - loc) / scale.

For positive concentrations, the distribution is equivalent to a hierarchical Exponential-Gamma model with X|rate ~ Exponential(rate) and rate ~ Gamma(concentration=1 / concentration, scale=scale / concentration). In the following, samp1 and samps2 are identically distributed:

genp = tfd.GeneralizedPareto(loc=0, scale=scale, concentration=conc)
samps1 = genp.sample(1000)
jd = tfd.JointDistributionNamed(dict(
    rate=tfd.Gamma(1 / genp.concentration, genp.scale / genp.concentration),
    x=lambda rate: tfd.Exponential(rate)))
samps2 = jd.sample(1000)['x']

The support of the distribution is always lower bounded by loc. When concentration < 0, the support is also upper bounded by loc + scale / abs(concentration).

Mathematical Details

The probability density function (pdf) is,

pdf(x; mu, sigma, shp, x > mu) =
    (1 + shp * (x - mu) / sigma)**(-1 / shp - 1) / sigma


  • concentration = shp, any real value,
  • scale = sigma, sigma > 0,
  • loc = mu.

The cumulative density function (cdf) is,

cdf(x; mu, sigma, shp, x > mu) = 1 - (1 + shp * (x - mu) / sigma)**(-1 / shp)

Distribution parameters are automatically broadcast in all functions; see examples for details.

Samples of this distribution are reparameterized (pathwise differentiable).


import tensorflow_probability as tfp
tfd = tfp.distributions

dist = tfd.GeneralizedPareto(loc=1., scale=2., concentration=0.03)
dist2 = tfd.GeneralizedPareto(loc=-2., scale=[3., 4.],
                              concentration=[[-.4], [0.2]])

Compute the gradients of samples w.r.t. the parameters:

loc = tf.Variable(3.0)
scale = tf.Variable(2.0)
conc = tf.Variable(0.1)
dist = tfd.GeneralizedPareto(loc, scale, conc)
with tf.GradientTape() as tape:
  samples = dist.sample(5)  # Shape [5]
  loss = tf.reduce_mean(tf.square(samples))  # Arbitrary loss function
# Unbiased stochastic gradients of the loss function
grads = tape.gradient(loss, dist.variables)

loc The location / shift of the distribution. GeneralizedPareto is a location-scale distribution. This parameter lower bounds the distribution's support. Must broadcast with scale, concentration. Floating point Tensor.
scale The scale of the distribution. GeneralizedPareto is a location-scale distribution, so doubling the scale doubles a sample and halves the density. Strictly positive floating point Tensor. Must broadcast with loc, concentration.
concentration The shape parameter of the distribution. The larger the magnitude, the more the distribution concentrates near loc (for concentration >= 0) or near loc - (scale/concentration) (for concentration < 0). Floating point Tensor.
validate_args Python bool, default False. When True distribution parameters are checked for validity despite possibly degrading runtime performance. When False invalid inputs may silently render incorrect outputs.
allow_nan_stats Python bool, default True. When True, statistics (e.g., mean, variance) use the value "NaN" to indicate the result is undefined. When False, an exception is raised if one or more of the statistic's batch members are undefined.
name Python str name prefixed to Ops created by this class.

TypeError if loc, scale, or concentration have different dtypes.

allow_nan_stats Python bool describing behavior when a stat is undefined.

Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.

batch_shape Shape of a single sample from a single event index as a TensorShape.

May be partially defined or unknown.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.


dtype The DType of Tensors handled by this Distribution.
event_shape Shape of a single sample from a single batch as a TensorShape.

May be partially defined or unknown.

experimental_shard_axis_names The list or structure of lists of active shard axis names.

name Name prepended to all ops created by this Distribution.
name_scope Returns a tf.name_scope instance for this class.
non_trainable_variables Sequence of non-trainable variables owned by this module and its submodules.

parameters Dictionary of parameters used to instantiate this Distribution.
reparameterization_type Describes how samples from the distribution are reparameterized.

Currently this is one of the static instances tfd.FULLY_REPARAMETERIZED or tfd.NOT_REPARAMETERIZED.


submodules Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

a = tf.Module()
b = tf.Module()
c = tf.Module()
a.b = b
b.c = c
list(a.submodules) == [b, c]
list(b.submodules) == [c]
list(c.submodules) == []

trainable_variables Sequence of trainable variables owned by this module and its submodules.

validate_args Python bool indicating possibly expensive checks are enabled.
variables Sequence of variables owned by this module and its submodules.



View source

Shape of a single sample from a single event index as a 1-D Tensor.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

name name to give to the op

batch_shape Tensor.


View source

Cumulative distribution function.

Given random variable X, the cumulative distribution function cdf is:

cdf(x) := P[X <= x]

value float or double Tensor.
name Python str prepended to names of ops created by this function.
**kwargs Named arguments forwarded to subclass implementation.

cdf a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.


View source

Creates a deep copy of the distribution.

**override_parameters_kwargs String/value dictionary of initialization arguments to override with new values.

distribution A new instance of type(self) initialized from the union of self.parameters and override_parameters_kwargs, i.e., dict(self.parameters, **override_parameters_kwargs).


View source


Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-k, vector-valued distribution, it is calculated as,

Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]

where Cov is a (batch of) k x k matrix, 0 <= (i, j) < k, and E denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), Covariance shall return a (batch of) matrices under some vectorization of the events, i.e.,

Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]