tf.distribute.experimental.coordinator.RemoteValue

View source on GitHub

An asynchronously available value of a scheduled function.

This class is used as the return value of tf.distribute.experimental.coordinator.ClusterCoordinator.schedule where the underlying value becomes available at a later time once the function has been executed.

Using tf.distribute.experimental.coordinator.RemoteValue as an input to a subsequent function scheduled with tf.distribute.experimental.coordinator.ClusterCoordinator.schedule is currently not supported.

Example:

strategy = tf.distribute.experimental.ParameterServerStrategy(
    cluster_resolver=...)
coordinator = (
    tf.distribute.experimental.coordinator.ClusterCoordinator(strategy))

with strategy.scope():
  v1 = tf.Variable(initial_value=0.0)
  v2 = tf.Variable(initial_value=1.0)

@tf.function
def worker_fn():
  v1.assign_add(0.1)
  v2.assign_sub(0.2)
  return v1.read_value() / v2.read_value()

result = coordinator.schedule(worker_fn)
# Note that `fetch()` gives the actual result instead of a `tf.Tensor`.
assert result.fetch() == 0.125

for _ in range(10):
  # `worker_fn` will be run on arbitrary workers that are available. The
  # `result` value will be available later.
  result = coordinator.schedule(worker_fn)

Methods

fetch

View source

fetch()

Wait for the result of RemoteValue to be ready and return the result.

This makes the value concrete by copying the remote value to local.