# tff.simulation.datasets.SqlClientData

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/federated/blob/v0.87.0 Version 2.0, January 2004 Licensed under the Apache License, Version 2.0 (the) |

A [`tff.simulation.datasets.ClientData`](https://www.tensorflow.org/federated/api_docs/python/tff/simulation/datasets/ClientData) backed by an SQL file.

Inherits From: [`ClientData`](https://www.tensorflow.org/federated/api_docs/python/tff/simulation/datasets/ClientData)

    tff.simulation.datasets.SqlClientData(
        database_filepath: str, split_name: Optional[str] = None
    )

This class expects that the SQL file has two tables: `examples` and
`client_metadata`.

Each row of the `examples` table corresponds to a sample in the dataset.
This table must contain at least the following three columns:

- `split_name`: `TEXT` column used to split test, holdout, and training examples.
- `client_id`: `TEXT` column identifying which user the example belongs to.
- `serialized_example_proto`: A serialized [`tf.train.Example`](https://www.tensorflow.org/api_docs/python/tf/train/Example) protocol buffer containing the example data.

Each row of the `client_metadata` table corresponds to a client in the
dataset. This table must contain at least the following three columns:

- `client_id`: `TEXT` column used to identify the client.
- `split_name`: `TEXT` column used to split test, holdout, and training examples.
- `num_examples`: `INTEGER` column containing the number of examples held by this client.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `database_filepath` | A `str` filepath to a SQL database. |
| `split_name` | An optional `str` identifier for the split of the database to use. This filters clients and examples based on the `split_name` column. A value of `None` means no filtering, selecting all examples. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `client_ids` | A list of string identifiers for clients in this dataset. |
| `dataset_computation` | A [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation) accepting a client ID, returning a dataset. <br /> > [!NOTE] > **Note:** the `dataset_computation` property is intended as a TFF-specific performance optimization for distributed execution. |
| `element_type_structure` | The element type information of the client datasets. elements returned by datasets in this `ClientData` object. |
| `serializable_dataset_fn` | A callable accepting a client ID and returning a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). <br /> Note that this callable must be traceable by TF, as it will be used in the context of a [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function). |

<br />

## Methods

### `create_tf_dataset_for_client`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    create_tf_dataset_for_client(
        client_id: str
    )

Creates a new [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) containing the client training examples.

This function will create a dataset for a given client if `client_id` is
contained in the `client_ids` property of the `SQLClientData`. Unlike
`self.serializable_dataset_fn`, this method is not serializable.

<br />

<br />

<br />

| Args ||
|---|---|
| `client_id` | The string identifier for the desired client. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) object. ||

<br />

### `create_tf_dataset_from_all_clients`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    create_tf_dataset_from_all_clients(
        seed: Optional[Union[int, Sequence[int]]] = None
    ) -> tf.data.Dataset

Creates a new [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) containing *all* client examples.

This function is intended for use training centralized, non-distributed
models (num_clients=1). This can be useful as a point of comparison
against federated models.

Currently, the implementation produces a dataset that contains
all examples from a single client in order, and so generally additional
shuffling should be performed.

<br />

<br />

<br />

| Args ||
|---|---|
| `seed` | Optional, a seed to determine the order in which clients are processed in the joined dataset. The seed can be any nonnegative 32-bit integer, an array of such integers, or `None`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) object. ||

<br />

### `datasets`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    datasets(
        limit_count: Optional[int] = None,
        seed: Optional[Union[int, Sequence[int]]] = None
    ) -> Iterable[tf.data.Dataset]

Yields the [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) for each client in random order.

This function is intended for use building a static array of client data
to be provided to the top-level federated computation.

<br />

<br />

<br />

| Args ||
|---|---|
| `limit_count` | Optional, a maximum number of datasets to return. |
| `seed` | Optional, a seed to determine the order in which clients are processed in the joined dataset. The seed can be any nonnegative 32-bit integer, an array of such integers, or `None`. |

<br />

### `from_clients_and_tf_fn`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    @classmethod
    from_clients_and_tf_fn(
        client_ids: Iterable[str],
        serializable_dataset_fn: Callable[[str], tf.data.Dataset]
    ) -> 'ClientData'

Constructs a `ClientData` based on the given function.

> [!WARNING]
> **Warning:** Because this function must be serializable within a [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function) and a [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation), it cannot be a [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation).

<br />

<br />

<br />

| Args ||
|---|---|
| `client_ids` | A non-empty list of strings to use as input to `create_dataset_fn`. |
| `serializable_dataset_fn` | A function that takes a client_id from the above list, and returns a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). This function must be serializable and usable within the context of a [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function) and [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation). |

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | If `serializable_dataset_fn` is a [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation). |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `ClientData` object. ||

<br />

### `preprocess`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    preprocess(
        preprocess_fn: Callable[[tf.data.Dataset], tf.data.Dataset]
    ) -> 'ClientData'

Applies `preprocess_fn` to each client's data.

<br />

<br />

<br />

| Args ||
|---|---|
| `preprocess_fn` | A callable accepting a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) and returning a preprocessed [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). This function must be traceable by TF. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A [`tff.simulation.datasets.ClientData`](https://www.tensorflow.org/federated/api_docs/python/tff/simulation/datasets/ClientData). ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `IncompatiblePreprocessFnError` | If `preprocess_fn` is a [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation). |

<br />

### `train_test_client_split`

[View source](https://github.com/tensorflow/federated/blob/v0.87.0
Version 2.0, January 2004
Licensed under the Apache License, Version 2.0 (the)

    @classmethod
    train_test_client_split(
        client_data: 'ClientData',
        num_test_clients: int,
        seed: Optional[Union[int, Sequence[int]]] = None
    ) -> tuple['ClientData', 'ClientData']

Returns a pair of (train, test) `ClientData`.

This method partitions the clients of `client_data` into two `ClientData`
objects with disjoint sets of [`ClientData.client_ids`](https://www.tensorflow.org/federated/api_docs/python/tff/simulation/datasets/ClientData#client_ids). All clients in the
test `ClientData` are guaranteed to have non-empty datasets, but the
training `ClientData` may have clients with no data.

> [!NOTE]
> **Note:** This method may be expensive, and so it may be useful to avoid calling multiple times and holding on to the results.

<br />

<br />

<br />

| Args ||
|---|---|
| `client_data` | The base `ClientData` to split. |
| `num_test_clients` | How many clients to hold out for testing. This can be at most len(client_data.client_ids) - 1, since we don't want to produce empty `ClientData`. |
| `seed` | Optional seed to fix shuffling of clients before splitting. The seed can be any nonnegative 32-bit integer, an array of such integers, or `None`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A pair (train_client_data, test_client_data), where test_client_data has `num_test_clients` selected at random, subject to the constraint they each have at least 1 batch in their dataset. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `ValueError` | If `num_test_clients` cannot be satistifed by `client_data`, or too many clients have empty datasets. |

<br />