# tff.learning.optimizers.Optimizer

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/federated/blob/v0.87.0 Version 2.0, January 2004 Licensed under the Apache License, Version 2.0 (the) |

Represents an optimizer for use in TensorFlow Federated.

Its pair of `initialize` and `next` methods define the optimization
algorithm, with `next` corresponding to a step of the optimizer.

This class captures iterative optimization algorithms where the same operation
is applied in every optimization step. The `next` method should be a
computation that can be implemented as a [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function). Each method will
generally only be traced once to create the corresponding TensorFlow graph
functions. Thus, the methods should not create or use [`tf.Variable`](https://www.tensorflow.org/api_docs/python/tf/Variable) objects.

Instead, any dependency between steps of the algorithm should be included
as tensors in a state. For instance, a momentum term for momentum SGD is
created in the `initialize` method as all-zeros tensor, which is then both
an input and an output of the `next` method. These aspects should be accessed
and changed via `get_hparams` and `set_hparams`, respectively.

As a best practice, any implementation using learning rate, should store it in
its state under the key [`tff.learning.optimizers.LEARNING_RATE_KEY`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/optimizers#LEARNING_RATE_KEY).

## Methods

### `get_hparams`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    get_hparams(
        state: State
    ) -> Hparams

Returns a dictionary containing the optimizer state hyperparameters.

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | The state of the optimizer. Must match the structure returned by the `initialize` method. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An ordered dictionary representing the hyperparameters in the given state. ||

<br />

### `initialize`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    @abc.abstractmethod
    initialize(
        specs: Any
    ) -> State

Returns the initial state of the optimizer.

<br />

<br />

<br />

| Args ||
|---|---|
| `specs` | A (possibly nested) structure of [`tf.TensorSpec`](https://www.tensorflow.org/api_docs/python/tf/TensorSpec)s describing the weights to be optimized. The `weights` and `grads` argument of `next` must match the structure and (shape, dtype) of `specs`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| Initial state of the optimizer. A (possibly nested) structure of tensors. ||

<br />

### `next`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    @abc.abstractmethod
    next(
        state: State, weights: Weights, gradients: Any
    ) -> tuple[State, Weights]

Takes a step of the optimizer.

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | State of the optimizer. A structure of tensors matching the structure returned by `initialize` method. |
| `weights` | The weights to be updated by the optimizer. A collection of tensors matching the structure of `specs` provided in the `initialize` method. |
| `gradients` | The gradients to use for the update by the optimizer. A collection of tensors matching the structure of `specs` provided in the `initialize` method. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A (state, weights) tuple representing the updated `state` and `weights`. ||

<br />

### `set_hparams`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    set_hparams(
        state: State, hparams: Hparams
    ) -> State

Returns an optimizer state with updated hyperparameters.

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | The state of the optimizer. Must match the structure returned by the `initialize` method. |
| `hparams` | A dictionary matching the output of `get_hparams` containing the updated hyperparameters to use. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An ordered dictionary representing the hyperparameters in the given state. ||

<br />