tf.VariableAggregation

Class VariableAggregation

Defined in tensorflow/python/ops/variables.py.

Indicates how a distributed variable will be aggregated.

tf.contrib.distribute.DistributionStrategy distributes a model by making multiple copies (called "towers") acting data-parallel on different elements of the input batch. When performing some variable-update operation, say var.assign_add(x), in a model, we need to resolve how to combine the different values for x computed in the different towers.

  • NONE: This is the default, giving an error if you use a variable-update operation with multiple towers.
  • SUM: Add the updates across towers.
  • MEAN: Take the arithmetic mean ("average") of the updates across towers.
  • ONLY_FIRST_TOWER: This is for when every tower is performing the same update, but we only want to perform the update once. Used, e.g., for the global step counter.

Class Members

MEAN

NONE

ONLY_FIRST_TOWER

SUM

__members__