<br />

<button value="stable" default="">See Stable</button> <button value="nightly">See Nightly</button>

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L532-L625) |

Distribution for data parallelism.

    tf.keras.distribution.DataParallel(
        device_mesh=None, devices=None, auto_shard_dataset=True
    )

You can choose to create this instance by either specifying
the `device_mesh` or `devices` arguments (but not both).

The `device_mesh` argument is expected to be a `DeviceMesh` instance,
and is expected to be 1D only. In case that the mesh has multiple axes,
then the first axis will be treated as the data parallel dimension
(and a warning will be raised).

When a list of `devices` are provided, they will be used to construct a
1D mesh.

When both `mesh` and `devices` are absent, then `list_devices()`
will be used to detect any available devices and create a 1D mesh from
them.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `device_mesh` | Optional `DeviceMesh` instance. |
| `devices` | Optional list of devices. |
| `auto_shard_dataset` | Automatically shard the dataset amongst processes in a multi-process setting. Set to `False` if the dataset is already sharded across hosts. Defaults to `True`. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `auto_shard_dataset` | <br /> <br /> |
| `batch_dim_name` | <br /> <br /> |
| `data_shard_id` | ID of the data shard for the current process. |
| `device_mesh` | <br /> <br /> |
| `num_model_replicas` | Number of model replicas. |
| `num_processes` | Total number of processes in the cluster. |

<br />

## Methods

### `distribute_dataset`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L425-L451)

    distribute_dataset(
        dataset
    )

Create a distributed dataset from the original global dataset.

<br />

<br />

<br />

| Args ||
|---|---|
| `dataset` | the original global dataset instance. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| If `auto_shard_dataset` is `True`, returns a sharded dataset that only produces data for the current local worker/process. Otherwise, returns the original dataset. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | if auto-sharding is requested in a multi-process setting, but the dataset type is not supported. |

<br />

### `distribute_tf_dataset`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L453-L523)

    distribute_tf_dataset(
        dataset
    )

### `get_data_layout`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L610-L613)

    get_data_layout(
        data_shape
    )

Retrieve the `TensorLayout` for the input data.

<br />

<br />

<br />

| Args ||
|---|---|
| `data_shape` | shape for the input data in list or tuple format. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| The `TensorLayout` for the data, which can be used by `backend.distribute_value()` to redistribute a input data. ||

<br />

### `get_tensor_layout`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L623-L625)

    get_tensor_layout(
        path
    )

Retrieve the `TensorLayout` for the intermediate tensor.

<br />

<br />

<br />

| Args ||
|---|---|
| `path` | a string path for the corresponding tensor. |

<br />

return:
The `TensorLayout` for the intermediate tensor, which can be used
by `backend.relayout()` to reshard the tensor. Could also return
None.

### `get_variable_layout`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L615-L621)

    get_variable_layout(
        variable
    )

Retrieve the `TensorLayout` for the variable.

<br />

<br />

<br />

| Args ||
|---|---|
| `variable` | A `Variable` instance. |

<br />

return:
The `TensorLayout` for the variable, which can be used by
`backend.distribute_value()` to redistribute a variable.

### `scope`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/distribution/distribution_lib.py#L399-L407)

    @contextlib.contextmanager
    scope()

Context manager to make the `Distribution` current.