# Module: tf.contrib.distribute

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/distribute/__init__.py) |

A distributed computation library for TF.

See [tensorflow/contrib/distribute/README.md](https://www.tensorflow.org/code/tensorflow/contrib/distribute/README.md)
for overview and examples.

## Classes

[`class AllReduceCrossDeviceOps`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/AllReduceCrossDeviceOps): Reduction using all-reduce.

[`class CollectiveAllReduceStrategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/CollectiveAllReduceStrategy): Distribution strategy that uses collective ops for all-reduce.

[`class CrossDeviceOps`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/CrossDeviceOps): Base class for cross-device reduction and broadcasting algorithms.

[`class DistributeConfig`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/DistributeConfig): A config tuple for distribution strategies.

[`class DistributionStrategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/Strategy): A list of devices with a state \& compute distribution policy.

[`class MirroredStrategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/MirroredStrategy): Mirrors vars to distribute across multiple devices and machines.

[`class Monitor`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/Monitor): Executes training steps, recovers and checkpoints.

[`class MultiWorkerAllReduce`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/MultiWorkerAllReduce): All-reduce algorithms for distributed TensorFlow.

[`class OneDeviceStrategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/OneDeviceStrategy): A distribution strategy for running on a single device.

[`class ParameterServerStrategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/ParameterServerStrategy): A parameter server DistributionStrategy.

[`class ReplicaContext`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/ReplicaContext): [`tf.distribute.Strategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/Strategy) API when in a replica context.

[`class StandardInputStep`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/StandardInputStep): Step with a standard implementation of input handling.

[`class StandardSingleLossStep`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/StandardSingleLossStep): A step function that implements a training step for a feed forward network.

[`class Step`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/Step): Interface for performing each step of a training algorithm.

[`class TPUStrategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/experimental/TPUStrategy): TPU distribution strategy implementation.

[`class UpdateContext`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/UpdateContext): Context manager when you are in `update()` or `update_non_slot()`.

## Functions

[`get_cross_replica_context(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/get_cross_replica_context): Returns the current tf.distribute.Strategy if in a cross-replica context.

[`get_distribution_strategy(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/get_strategy): Returns the current [`tf.distribute.Strategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/Strategy) object.

[`get_loss_reduction(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/get_loss_reduction): [`tf.distribute.ReduceOp`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/ReduceOp) corresponding to the last loss reduction.

[`get_replica_context(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/get_replica_context): Returns the current [`tf.distribute.ReplicaContext`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/ReplicaContext) or `None`.

[`get_strategy(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/get_strategy): Returns the current [`tf.distribute.Strategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/Strategy) object.

[`has_distribution_strategy(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/has_strategy): Return if there is a current non-default [`tf.distribute.Strategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/Strategy).

[`has_strategy(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/has_strategy): Return if there is a current non-default [`tf.distribute.Strategy`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/Strategy).

[`in_cross_replica_context(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/in_cross_replica_context): Returns `True` if in a cross-replica context.

[`initialize_tpu_system(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/tpu/experimental/initialize_tpu_system): Initialize the TPU devices.

[`require_replica_context(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/require_replica_context): Verify in `replica_ctx` replica context.

[`run_standard_tensorflow_server(...)`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/distribute/run_standard_tensorflow_server): Starts a standard TensorFlow server.