tff.learning.optimizers.build_adagrad

Returns a tff.learning.optimizers.Optimizer for Adagrad.

tff.learning.optimizers.build_adagrad(
    learning_rate: optimizer.Float,
    initial_preconditioner_value: optimizer.Float = 0.1,
    epsilon: optimizer.Float = 1e-07
) -> tff.learning.optimizers.Optimizer

The Adagrad optimizer is based on Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

The update rule given learning rate lr, epsilon eps, preconditioner s, weights w and gradients g is:

s = s + g**2
w = w - lr * g / sqrt(s + eps)

learning_rate A positive float for learning rate.
initial_preconditioner_value A non-negative float, initial value for the preconditioner.
epsilon A small non-negative float, used to maintain numerical stability.