Indicates how a distributed variable will be aggregated.
tf.contrib.distribute.DistributionStrategy distributes a model by making
multiple copies (called "towers") acting data-parallel on different elements
of the input batch. When performing some variable-update operation, say
var.assign_add(x), in a model, we need to resolve how to combine the
different values for
x computed in the different towers.
NONE: This is the default, giving an error if you use a variable-update operation with multiple towers.
SUM: Add the updates across towers.
MEAN: Take the arithmetic mean ("average") of the updates across towers.
ONLY_FIRST_TOWER: This is for when every tower is performing the same update, but we only want to perform the update once. Used, e.g., for the global step counter.