tf.contrib.distribute.ReductionToOneDeviceCrossTowerOps

Class ReductionToOneDeviceCrossTowerOps

Inherits From: CrossTowerOps

Always do reduction to one device first and then do broadcasting.

Batch reduction is done by reduction on each element one by one.

Methods

__init__

__init__(
reduce_to_device=None,
)


Constructor.

Args:

• reduce_to_device: the intermediate device to reduce to. If None, reduce to the first device in destinations of the reduce() method.
• accumulation_fn: a function that does accumulation.

batch_reduce

batch_reduce(
method_string,
value_destination_pairs
)


Reduce PerDevice objects in a batch.

Reduce each first element in value_destination_pairs to each second element which indicates the destinations.

Args:

• method_string: either 'sum' or 'mean' specifying the reduction method.
• value_destination_pairs: a list or a tuple of tuples of PerDevice objects and destinations. If a destination is None, then the destinations are set to match the devices of the input PerDevice object.

Returns:

a list of Mirrored objects.

Raises:

• ValueError: if value_destination_pairs is not a list or a tuple of tuples of PerDevice objects and destinations

broadcast

broadcast(
tensor,
destinations
)


Broadcast the tensor to destinations.

Args:

• tensor: the tensor to broadcast.
• destinations: the broadcast destinations.

Returns:

a Mirrored object.

reduce

reduce(
method_string,
per_device_value,
destinations=None
)


Reduce per_device_value to destinations.

It runs the reduction operation defined by method_string and put the result on destinations.

Args:

• method_string: either 'sum' or 'mean' specifying the reduction method.
• per_device_value: a PerDevice object.
• destinations: the reduction destinations.

Returns:

a Mirrored object.

Raises:

• ValueError: if per_device_value is not a PerDevice object.