Optimizer that implements the Adam algorithm.

Inherits From: `Optimizer`

``````tf.keras.optimizers.Adam(
)
``````

### Used in the notebooks

Adam optimization is a stochastic gradient descent method that is based on adaptive estimation of first-order and second-order moments. According to the paper Adam: A Method for Stochastic Optimization. Kingma et al., 2014, the method is "computationally efficient, has little memory requirement, invariant to diagonal rescaling of gradients, and is well suited for problems that are large in terms of data/parameters".

For AMSGrad see On The Convergence Of Adam And Beyond. Reddi et al., 5-8.

#### Args:

• `learning_rate`: A Tensor or a floating point value. The learning rate.
• `beta_1`: A float value or a constant float tensor. The exponential decay rate for the 1st moment estimates.
• `beta_2`: A float value or a constant float tensor. The exponential decay rate for the 2nd moment estimates.
• `epsilon`: A small constant for numerical stability. This epsilon is "epsilon hat" in the Kingma and Ba paper (in the formula just before Section 2.1), not the epsilon in Algorithm 1 of the paper.
• `amsgrad`: boolean. Whether to apply AMSGrad variant of this algorithm from the paper "On the Convergence of Adam and beyond".
• `name`: Optional name for the operations created when applying gradients. Defaults to "Adam".
• `**kwargs`: keyword arguments. Allowed to be {`clipnorm`, `clipvalue`, `lr`, `decay`}. `clipnorm` is clip gradients by norm; `clipvalue` is clip gradients by value, `decay` is included for backward compatibility to allow time inverse decay of learning rate. `lr` is included for backward compatibility, recommended to use `learning_rate` instead.

#### Attributes:

• `iterations`: Variable. The number of training steps this Optimizer has run.
• `weights`: Returns variables of this Optimizer based on the order created.

## Methods

### `add_slot`

View source

``````add_slot(
var, slot_name, initializer='zeros'
)
``````

Add a new slot variable for `var`.

### `add_weight`

View source

``````add_weight(
name, shape, dtype=None, initializer='zeros', trainable=None,
synchronization=tf.VariableSynchronization.AUTO,
aggregation=tf.compat.v1.VariableAggregation.NONE
)
``````

### `apply_gradients`

View source

``````apply_gradients(
)
``````

This is the second part of `minimize()`. It returns an `Operation` that applies gradients.

#### Args:

• `grads_and_vars`: List of (gradient, variable) pairs.
• `name`: Optional name for the returned operation. Default to the name passed to the `Optimizer` constructor.

#### Returns:

An `Operation` that applies the specified gradients. The `iterations` will be automatically increased by 1.

#### Raises:

• `TypeError`: If `grads_and_vars` is malformed.
• `ValueError`: If none of the variables have gradients.

### `from_config`

View source

``````@classmethod
from_config(
cls, config, custom_objects=None
)
``````

Creates an optimizer from its config.

This method is the reverse of `get_config`, capable of instantiating the same optimizer from the config dictionary.

#### Arguments:

• `config`: A Python dictionary, typically the output of get_config.
• `custom_objects`: A Python dictionary mapping names to additional Python objects used to create this optimizer, such as a function used for a hyperparameter.

#### Returns:

An optimizer instance.

### `get_config`

View source

``````get_config()
``````

Returns the config of the optimimizer.

An optimizer config is a Python dictionary (serializable) containing the configuration of an optimizer. The same optimizer can be reinstantiated later (without any saved state) from this configuration.

#### Returns:

Python dictionary.

### `get_gradients`

View source

``````get_gradients(
loss, params
)
``````

Returns gradients of `loss` with respect to `params`.

#### Arguments:

• `loss`: Loss tensor.
• `params`: List of variables.

#### Raises:

• `ValueError`: In case any gradient cannot be computed (e.g. if gradient function not implemented).

### `get_slot`

View source

``````get_slot(
var, slot_name
)
``````

### `get_slot_names`

View source

``````get_slot_names()
``````

A list of names for this optimizer's slots.

### `get_updates`

View source

``````get_updates(
loss, params
)
``````

### `get_weights`

View source

``````get_weights()
``````

### `minimize`

View source

``````minimize(
)
``````

Minimize `loss` by updating `var_list`.

This method simply computes gradient using `tf.GradientTape` and calls `apply_gradients()`. If you want to process the gradient before applying then call `tf.GradientTape` and `apply_gradients()` explicitly instead of using this function.

#### Args:

• `loss`: A callable taking no arguments which returns the value to minimize.
• `var_list`: list or tuple of `Variable` objects to update to minimize `loss`, or a callable returning the list or tuple of `Variable` objects. Use callable when the variable list would otherwise be incomplete before `minimize` since the variables are created at the first time `loss` is called.
• `grad_loss`: Optional. A `Tensor` holding the gradient computed for `loss`.
• `name`: Optional name for the returned operation.

#### Returns:

An `Operation` that updates the variables in `var_list`. The `iterations` will be automatically increased by 1.

#### Raises:

• `ValueError`: If some of the variables are not `Variable` objects.

### `set_weights`

View source

``````set_weights(
weights
)
``````

### `variables`

View source

``````variables()
``````

Returns variables of this Optimizer based on the order created.