DistributionStrategy API inside a
__init__( distribution_strategy, tower_id )
The device this tower is to be executed on, as a string.
Returns whether there is a single tower or multiple.
Returns number of towers, for purposes of averaging across towers.
Which tower is being defined, a number from 0 to
num_towers - 1.
__exit__( exception_type, exception_value, traceback )
merge_call( merge_fn, *args, **kwargs )
Merge args across towers and run
merge_fn in a cross-tower context.
This allows communication and coordination when there are multiple calls
to a model function triggered by a call to
MirroredDistribution.call_for_each_tower() for an explanation.
Otherwise, this is equivalent to:
distribution = get_distribution_strategy() with cross-tower-context(distribution): return merge_fn(distribution, *args, **kwargs)
merge_fn: function that joins arguments from threads that are given as PerDevice. It accepts
DistributionStrategyobject as the first argument.
*args: positional per-thread arguments for
**kwargs: keyword per-thread arguments for
The return value of
merge_fn, except for
PerDevice values which are