tf.distribute.ReductionToOneDevice

A CrossDeviceOps implementation that copies values to one device to reduce.

Inherits From: CrossDeviceOps

This implementation always copies values to one device to reduce them, then broadcast reduced values to the destinations. It doesn't support efficient batching.

Here is how you can use ReductionToOneDevice in tf.distribute.MirroredStrategy:

  strategy = tf.distribute.MirroredStrategy(
    cross_device_ops=tf.distribute.ReductionToOneDevice())

reduce_to_device the intermediate device to reduce to. If None, reduce to the first device in