# tf.test.experimental.sync_devices

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v2.13.1/tensorflow/python/framework/test_util.py#L4023-L4076) |

Synchronizes all devices.

#### View aliases


**Compat aliases for migration**

See
[Migration guide](https://www.tensorflow.org/guide/migrate) for
more details.

[`tf.compat.v1.test.experimental.sync_devices`](https://www.tensorflow.org/api_docs/python/tf/test/experimental/sync_devices)

<br />

    tf.test.experimental.sync_devices()

By default, GPUs run asynchronously. This means that when you run an op on the
GPU, like [`tf.linalg.matmul`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/linalg/matmul), the op may still be running on the GPU when the
function returns. Non-GPU devices can also be made to run asynchronously by
calling [`tf.config.experimental.set_synchronous_execution(False)`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/config/experimental/set_synchronous_execution). Calling
`sync_devices()` blocks until pending ops have finished executing. This is
primarily useful for measuring performance during a benchmark.

For example, here is how you can measure how long [`tf.linalg.matmul`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/linalg/matmul) runs:

    import time
    x = tf.random.normal((4096, 4096))
    tf.linalg.matmul(x, x)  # Warmup.
    tf.test.experimental.sync_devices()  # Block until warmup has completed.

    start = time.time()
    y = tf.linalg.matmul(x, x)
    tf.test.experimental.sync_devices()  # Block until matmul has completed.
    end = time.time()
    print(f'Time taken: {end - start}')

If the call to `sync_devices()` was omitted, the time printed could be too
small. This is because the op could still be running asynchronously when
the line `end = time.time()` is executed.

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `RuntimeError` | If run outside Eager mode. This must be called in Eager mode, outside any [`tf.function`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/function)s. |

<br />