A LearningRateSchedule that uses an inverse time decay schedule.

Inherits From: LearningRateSchedule

Used in the notebooks

Used in the tutorials

When training a model, it is often useful to lower the learning rate as the training progresses. This schedule applies the inverse decay function to an optimizer step, given a provided initial learning rate. It requires a step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step.

The schedule a 1-arg callable that produces a decayed learning rate when passed the current optimizer step. This can be useful for changing the learning rate value across different invocations of optimizer functions. It is computed as:

def decayed_learning_rate(step):
  return initial_learning_rate / (1 + decay_rate * step / decay_step)

or, if