# tf.distribute.CrossDeviceOps

<br />

|---|---|
| [![](https://www.tensorflow.org/images/tf_logo_32px.png) TensorFlow 1 version](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distribute/CrossDeviceOps) | [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L237-L403) |

Base class for cross-device reduction and broadcasting algorithms.

#### View aliases


**Compat aliases for migration**

See
[Migration guide](https://www.tensorflow.org/guide/migrate) for
more details.

[`tf.compat.v1.distribute.CrossDeviceOps`](https://www.tensorflow.org/api_docs/python/tf/distribute/CrossDeviceOps)

<br />

    tf.distribute.CrossDeviceOps()

## Methods

### `batch_reduce`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L284-L327)

    batch_reduce(
        reduce_op, value_destination_pairs
    )

Reduce PerReplica objects in a batch.

Reduce each first element in `value_destination_pairs` to each second
element which indicates the destinations.

This can be faster than multiple individual `reduce`s because we can
fuse several tensors into one or multiple packs before reduction.

<br />

<br />

<br />

| Args ||
|---|---|
| `reduce_op` | An instance of [`tf.distribute.ReduceOp`](https://www.tensorflow.org/versions/r2.1/api_docs/python/tf/distribute/ReduceOp) that indicates how the `per_replica_value` will be reduced. |
| `value_destination_pairs` | a list or a tuple of PerReplica objects (or tensors with device set if there is one device) and destinations. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| a list of Mirrored objects. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | if `value_destination_pairs` is not an iterable of tuples of PerReplica objects and destinations. |

<br />

### `batch_reduce_implementation`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L367-L390)

    batch_reduce_implementation(
        reduce_op, value_destination_pairs
    )

Implementation of reduce PerReplica objects in a batch.

Overriding this method is useful for subclass implementers.

Reduce each first element in `value_destination_pairs` to each second
element which indicates the destinations.

<br />

<br />

<br />

| Args ||
|---|---|
| `reduce_op` | An instance of [`tf.distribute.ReduceOp`](https://www.tensorflow.org/versions/r2.1/api_docs/python/tf/distribute/ReduceOp) that indicates how per_replica_value will be reduced. |
| `value_destination_pairs` | an iterable of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| a list of Mirrored objects. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | if `value_destination_pairs` is not an iterable of tuples of PerReplica objects and destinations |

<br />

### `broadcast`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L329-L340)

    broadcast(
        tensor, destinations
    )

Broadcast the `tensor` to destinations.

<br />

<br />

<br />

| Args ||
|---|---|
| `tensor` | the tensor to broadcast. |
| `destinations` | the broadcast destinations. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| a Mirrored object. ||

<br />

### `broadcast_implementation`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L392-L403)

    broadcast_implementation(
        tensor, destinations
    )

Implementation of broadcast the `tensor` to destinations.

<br />

<br />

<br />

| Args ||
|---|---|
| `tensor` | the tensor to broadcast. |
| `destinations` | the broadcast destinations. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| a Mirrored object. ||

<br />

### `reduce`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L248-L282)

    reduce(
        reduce_op, per_replica_value, destinations
    )

Reduce `per_replica_value` to `destinations`.

It runs the reduction operation defined by `reduce_op` and put the
result on `destinations`.

<br />

<br />

<br />

| Args ||
|---|---|
| `reduce_op` | An instance of [`tf.distribute.ReduceOp`](https://www.tensorflow.org/versions/r2.1/api_docs/python/tf/distribute/ReduceOp) that indicates how per_replica_value will be reduced. |
| `per_replica_value` | a PerReplica object or a tensor with device set. |
| `destinations` | the reduction destinations. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| a Mirrored object. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | if per_replica_value can't be converted to a PerReplica object or if destinations aren't strings, Variables or DistributedValues |

<br />

### `reduce_implementation`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/distribute/cross_device_ops.py#L342-L365)

    reduce_implementation(
        reduce_op, per_replica_value, destinations
    )

The implementation of reduce of `per_replica_value` to `destinations`.

Overriding this method is useful for subclass implementers.

It runs the reduction operation defined by `reduce_op` and put the
result on `destinations`.

<br />

<br />

<br />

| Args ||
|---|---|
| `reduce_op` | An instance [`tf.distribute.ReduceOp`](https://www.tensorflow.org/versions/r2.1/api_docs/python/tf/distribute/ReduceOp) that indicates of how per_replica_value will be reduced. |
| `per_replica_value` | a PerReplica object or a tensor with device set. |
| `destinations` | the reduction destinations. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| a Mirrored object. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | if per_replica_value can't be converted to a PerReplica object. |

<br />