# tf.contrib.estimator.TowerOptimizer

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py) |

Gathers gradients from all towers and reduces them in the last one.

Inherits From: [`Optimizer`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/train/Optimizer)

    tf.contrib.estimator.TowerOptimizer(
        optimizer_or_optimizer_fn
    )

<br />

<br />

<br />

| ## Args ||
|---|---|
| `optimizer_or_optimizer_fn` | an instance of optimizer to wrap. That instance is going to be used for optimizer-specific logic. This can also be a no-argument function that returns such an optimizer instance. |

<br />

## Methods

### `apply_gradients`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    apply_gradients(
        grads_and_vars, global_step=None, **kwargs
    )

Collect gradients updates to apply them with the last tower.

### `compute_gradients`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    compute_gradients(
        loss, *args, **kwargs
    )

Compute gradients, but first, if needed, scale the loss.

### `get_name`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    get_name(
        *args, **kwargs
    )

### `get_slot`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    get_slot(
        *args, **kwargs
    )

Return a slot named `name` created for `var` by the Optimizer.

Some `Optimizer` subclasses use additional variables. For example
`Momentum` and `Adagrad` use variables to accumulate updates. This method
gives access to these `Variable` objects if for some reason you need them.

Use `get_slot_names()` to get the list of slot names created by the
`Optimizer`.

<br />

<br />

<br />

| Args ||
|---|---|
| `var` | A variable passed to `minimize()` or `apply_gradients()`. |
| `name` | A string. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| The `Variable` for the slot if it was created, `None` otherwise. ||

<br />

### `get_slot_names`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    get_slot_names(
        *args, **kwargs
    )

Return a list of the names of slots created by the `Optimizer`.

See `get_slot()`.

<br />

<br />

<br />

| Returns ||
|---|---|
| A list of strings. ||

<br />

### `has_been_used`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    @staticmethod
    has_been_used()

### `minimize`

[View source](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/training/optimizer.py#L355-L413)

    minimize(
        loss, global_step=None, var_list=None, gate_gradients=GATE_OP,
        aggregation_method=None, colocate_gradients_with_ops=False, name=None,
        grad_loss=None
    )

Add operations to minimize `loss` by updating `var_list`.

This method simply combines calls `compute_gradients()` and
`apply_gradients()`. If you want to process the gradient before applying
them call `compute_gradients()` and `apply_gradients()` explicitly instead
of using this function.

<br />

<br />

<br />

| Args ||
|---|---|
| `loss` | A `Tensor` containing the value to minimize. |
| `global_step` | Optional `Variable` to increment by one after the variables have been updated. |
| `var_list` | Optional list or tuple of `Variable` objects to update to minimize `loss`. Defaults to the list of variables collected in the graph under the key `GraphKeys.TRAINABLE_VARIABLES`. |
| `gate_gradients` | How to gate the computation of gradients. Can be `GATE_NONE`, `GATE_OP`, or `GATE_GRAPH`. |
| `aggregation_method` | Specifies the method used to combine gradient terms. Valid values are defined in the class `AggregationMethod`. |
| `colocate_gradients_with_ops` | If True, try colocating gradients with the corresponding op. |
| `name` | Optional name for the returned operation. |
| `grad_loss` | Optional. A `Tensor` holding the gradient computed for `loss`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An Operation that updates the variables in `var_list`. If `global_step` was not `None`, that operation also increments `global_step`. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | If some of the variables are not `Variable` objects. |

<br />

#### Eager Compatibility

When eager execution is enabled, `loss` should be a Python function that
takes no arguments and computes the value to be minimized. Minimization (and
gradient computation) is done with respect to the elements of `var_list` if
not None, else with respect to any trainable variables created during the
execution of the `loss` function. `gate_gradients`, `aggregation_method`,
`colocate_gradients_with_ops` and `grad_loss` are ignored when eager
execution is enabled.

### `variables`

[View source](https://github.com/tensorflow/estimator/tree/master/tensorflow_estimator/contrib/estimator/python/estimator/replicate_model_fn.py)

    variables(
        *args, **kwargs
    )

A list of variables which encode the current state of `Optimizer`.

Includes slot variables and additional global variables created by the
optimizer in the current default graph.

<br />

<br />

<br />

| Returns ||
|---|---|
| A list of variables. ||

<br />

## Class Variables

- `COLLECTION_FOR_GRAPH_STATES = 'replicate_model_fn_graph_states'`
- `GATE_GRAPH = 2`
- `GATE_NONE = 0`
- `GATE_OP = 1`