tf.contrib.distribute.CrossTowerOps

Class CrossTowerOps

Defined in tensorflow/contrib/distribute/python/cross_tower_ops.py.

Base class for cross-tower reduction and broadcasting algorithms.

Methods

__init__

__init__()

batch_reduce

batch_reduce(
    aggregation,
    value_destination_pairs
)

Reduce PerDevice objects in a batch.

Reduce each first element in value_destination_pairs to each second element which indicates the destinations.

Args:

  • aggregation: Indicates how a variable will be aggregated. Accepted values are tf.VariableAggregation.SUM, tf.VariableAggregation.MEAN.
  • value_destination_pairs: a list or a tuple of tuples of PerDevice objects and destinations. If a destination is None, then the destinations are set to match the devices of the input PerDevice object.

Returns:

a list of Mirrored objects.

Raises:

  • ValueError: if value_destination_pairs is not a list or a tuple of tuples of PerDevice objects and destinations

broadcast

broadcast(
    tensor,
    destinations
)

Broadcast the tensor to destinations.

Args:

  • tensor: the tensor to broadcast.
  • destinations: the broadcast destinations.

Returns:

a Mirrored object.

reduce

reduce(
    aggregation,
    per_device_value,
    destinations=None
)

Reduce per_device_value to destinations.

It runs the reduction operation defined by aggregation and put the result on destinations.

Args:

Returns:

a Mirrored object.

Raises:

  • ValueError: if per_device_value is not a PerDevice object.