# tff.learning.optimizers.build_yogi

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/federated/blob/v0.87.0 Version 2.0, January 2004 Licensed under the Apache License, Version 2.0 (the) |

Returns a [`tff.learning.optimizers.Optimizer`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/optimizers/Optimizer) for Yogi.

    tff.learning.optimizers.build_yogi(
        learning_rate: optimizer.Float,
        beta_1: optimizer.Float = 0.9,
        beta_2: optimizer.Float = 0.999,
        epsilon: optimizer.Float = 0.001,
        initial_preconditioner_value: optimizer.Float = 1e-06
    ) -> https://www.tensorflow.org/federated/api_docs/python/tff/learning/optimizers/Optimizer

The Yogi optimizer is based on [Adaptive methods for nonconvex optimization](https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization.pdf)

The update rule given learning rate `lr`, epsilon `eps`, accumulator `acc`,
preconditioner `s`, iteration `t`, weights `w` and gradients `g` is:

    acc = beta_1 * acc + (1 - beta_1) * g
    s = s + (1 - beta_2) * sign(g - s) * (g ** 2)
    normalized_lr = lr * sqrt(1 - beta_2**t) / (1 - beta_1**t)
    w = w - normalized_lr * acc / (sqrt(s) + eps)

Implementation of Yogi is based on additive updates, as opposed to
multiplicative updates (as in Adam). Experiments show better performance
across NLP and Vision tasks both in centralized and federated settings.

Typically use 10x the learning rate used for Adam.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `learning_rate` | A positive `float` for learning rate. |
| `beta_1` | A `float` between `0.0` and `1.0` for the decay used to track the previous gradients. |
| `beta_2` | A `float` between `0.0` and `1.0` for the decay used to track the magnitude (second moment) of previous gradients. |
| `epsilon` | A constant trading off adaptivity and noise.. |
| `initial_preconditioner_value` | The starting value for preconditioner. Only positive values are allowed. |

<br />