Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tf_agents.utils.common.ornstein_uhlenbeck_process

View source on GitHub

An op for generating noise from a zero-mean Ornstein-Uhlenbeck process.

tf_agents.utils.common.ornstein_uhlenbeck_process(
    initial_value, damping=0.15, stddev=0.2, seed=None,
    scope='ornstein_uhlenbeck_noise'
)

The Ornstein-Uhlenbeck process is a process that generates temporally correlated noise via a random walk with damping. This process describes the velocity of a particle undergoing brownian motion in the presence of friction. This can be useful for exploration in continuous action environments with momentum.

The temporal update equation is: x_next = (1 - damping) * x + N(0, std_dev)

Args:

  • initial_value: Initial value of the process.
  • damping: The rate at which the noise trajectory is damped towards the mean. We must have 0 <= damping <= 1, where a value of 0 gives an undamped random walk and a value of 1 gives uncorrelated Gaussian noise. Hence in most applications a small non-zero value is appropriate.
  • stddev: Standard deviation of the Gaussian component.
  • seed: Seed for random number generation.
  • scope: Scope of the variables.

Returns:

An op that generates noise.