Google I/O returns May 18-20! Reserve space and build your schedule Register now


Returns a decayed value of init_value over time.

When training a model with a regularizer, the objective function can be formulated as the following:

$$objective = \lambda_1 * loss + \lambda_2 * regularization$$

This function can be used for three cases:

  1. Incrementally diminishing the importance of the loss term, by applying a decay function to the

over time. We'll denote this by writing


= decay_over_time(init_value).

  1. Incrementally increasing the importance of the regularization term, by setting

= init_value - decay_over_time(init_value).

  1. Combining the above two cases, namely, setting

= decay_over_time(init_value) and


= init_value - decay_over_time(init_value).

This function requires a global_step value to compute the decayed value.

global_step A scalar int32 or int64 Tensor or a Python number. Must be positive.
decay_config A nsl.configs.DecayConfig for computing the decay value.
init_value A scalar Tensor to set the initial value to be decayed.

A scalar float Tensor.