Optimizer that implements the RMSprop algorithm.

Inherits From: Optimizer

Used in the notebooks

Used in the guide Used in the tutorials

The gist of RMSprop is to:

  • Maintain a moving (discounted) average of the square of gradients
  • Divide the gradient by the root of this average

This implementation of RMSprop uses plain momentum, not Nesterov momentum.

The centered version additionally maintains a moving average of the gradients, and uses that average to estimate the variance.

learning_rate A Tensor, floating point value, or a schedule that is a tf.keras.optimizers.schedules.LearningRateSchedule, or a callable that takes no arguments and returns the actual value to use. The learning rate. Defeaults to 0.001.
rho Discounting factor for the history/coming gradient. Defaults to 0.9.
momentum A scalar or a scalar Tensor. Defaults to 0.0.
epsilon A small constant for numerical st