{ }
Update '*var' according to the Adam algorithm.
tf.raw_ops.ResourceApplyAdamWithAmsgrad(
var,
m,
v,
vhat,
beta1_power,
beta2_power,
lr,
beta1,
beta2,
epsilon,
grad,
use_locking=False,
name=None
)
\[\text{lr}_t := \mathrm{learning_rate} * \sqrt{1 - \beta_2^t} / (1 - \beta_1^t)\]
\[m_t := \beta_1 * m_{t-1} + (1 - \beta_1) * g\]
\[v_t := \beta_2 * v_{t-1} + (1 - \beta_2) * g * g\]
\[\hat{v}_t := max{\hat{v}_{t-1}, v_t}\]
\[\text{variable} := \text{variable} - \text{lr}_t * m_t / (\sqrt{\hat{v}_t} + \epsilon)\]
Returns | |
---|---|
The created Operation. |