# Module: tf.contrib.opt

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/opt/__init__.py) |

A module containing optimization routines.

## Classes

[`class AGNCustomGetter`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/AGNCustomGetter): Custom_getter class is used to do:

[`class AGNOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/AGNOptimizer): Wrapper that implements the Accumulated GradientNormalization algorithm.

[`class AdaMaxOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/AdaMaxOptimizer): Optimizer that implements the AdaMax algorithm.

[`class AdamGSOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/AdamGSOptimizer): Optimizer that implements the Adam algorithm.

[`class AdamWOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/AdamWOptimizer): Optimizer that implements the Adam algorithm with weight decay.

[`class AddSignOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/AddSignOptimizer): Optimizer that implements the AddSign update.

[`class DecoupledWeightDecayExtension`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/DecoupledWeightDecayExtension): This class allows to extend optimizers with decoupled weight decay.

[`class DropStaleGradientOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/DropStaleGradientOptimizer): Wrapper optimizer that checks and drops stale gradient.

[`class ElasticAverageCustomGetter`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ElasticAverageCustomGetter): Custom_getter class is used to do:

[`class ElasticAverageOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ElasticAverageOptimizer): Wrapper optimizer that implements the Elastic Average SGD algorithm.

[`class ExternalOptimizerInterface`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ExternalOptimizerInterface): Base class for interfaces with external optimization algorithms.

[`class GGTOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/GGTOptimizer): Optimizer that implements the GGT algorithm.

[`class LARSOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/LARSOptimizer): Layer-wise Adaptive Rate Scaling for large batch training.

[`class LazyAdamGSOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/LazyAdamGSOptimizer): Variant of the Adam optimizer that handles sparse updates more efficiently.

[`class LazyAdamOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/LazyAdamOptimizer): Variant of the Adam optimizer that handles sparse updates more efficiently.

[`class ModelAverageCustomGetter`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ModelAverageCustomGetter): Custom_getter class is used to do.

[`class ModelAverageOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ModelAverageOptimizer): Wrapper optimizer that implements the Model Average algorithm.

[`class MomentumWOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/MomentumWOptimizer): Optimizer that implements the Momentum algorithm with weight_decay.

[`class MovingAverageOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/MovingAverageOptimizer): Optimizer that computes a moving average of the variables.

[`class MultitaskOptimizerWrapper`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/MultitaskOptimizerWrapper): Optimizer wrapper making all-zero gradients harmless.

[`class NadamOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/NadamOptimizer): Optimizer that implements the Nadam algorithm.

[`class PowerSignOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/PowerSignOptimizer): Optimizer that implements the PowerSign update.

[`class RegAdagradOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/RegAdagradOptimizer): RegAdagrad: Adagrad with updates that optionally skip updating the slots.

[`class ScipyOptimizerInterface`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ScipyOptimizerInterface): Wrapper allowing `scipy.optimize.minimize` to operate a [`tf.compat.v1.Session`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/Session).

[`class ShampooOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/ShampooOptimizer): The Shampoo Optimizer

[`class VariableClippingOptimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/VariableClippingOptimizer): Wrapper optimizer that clips the norm of specified variables after update.

## Functions

[`clip_gradients_by_global_norm(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/clip_gradients_by_global_norm): Clips gradients of a multitask loss by their global norm.

[`extend_with_decoupled_weight_decay(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/opt/extend_with_decoupled_weight_decay): Factory function returning an optimizer class with decoupled weight decay.