Thanks for tuning in to Google I/O. View all sessions on demand

# nsl.lib.decay_over_time

Returns a decayed value of init_value over time.

When training a model with a regularizer, the objective function can be formulated as the following:

$objective = \lambda_1 * loss + \lambda_2 * regularization$

This function can be used for three cases:

1. Incrementally diminishing the importance of the loss term, by applying a decay function to the $$\lambda_1$$ over time. We'll denote this by writing $$\lambda_1$$ = decay_over_time(init_value).
2. Incrementally increasing the importance of the regularization term, by setting $$\lambda_2$$ = init_value - decay_over_time(init_value).
3. Combining the above two cases, namely, setting $$\lambda_1$$ = decay_over_time(init_value) and $$\lambda_2$$ = init_value - decay_over_time(init_value).

This function requires a global_step value to compute the decayed value.

global_step A scalar int32 or int64 Tensor or a Python number. Must be positive.
decay_config A nsl.configs.DecayConfig for computing the decay value.
init_value A scalar Tensor to set the initial value to be decayed.

A scalar float Tensor.

[{ "type": "thumb-down", "id": "missingTheInformationINeed", "label":"Missing the information I need" },{ "type": "thumb-down", "id": "tooComplicatedTooManySteps", "label":"Too complicated / too many steps" },{ "type": "thumb-down", "id": "outOfDate", "label":"Out of date" },{ "type": "thumb-down", "id": "samplesCodeIssue", "label":"Samples / code issue" },{ "type": "thumb-down", "id": "otherDown", "label":"Other" }]
[{ "type": "thumb-up", "id": "easyToUnderstand", "label":"Easy to understand" },{ "type": "thumb-up", "id": "solvedMyProblem", "label":"Solved my problem" },{ "type": "thumb-up", "id": "otherUp", "label":"Other" }]