tff.simulation.FilePerUserClientData

View source on GitHub

Class FilePerUserClientData

A tf.simulation.ClientData that maps a set of files to a dataset.

Inherits From: ClientData

This mapping is restricted to one file per user.

__init__

View source

__init__(
    client_ids,
    create_tf_dataset_fn
)

Constructs a tf.simulation.ClientData object.

Args:

• client_ids: A list of client_ids.
• create_tf_dataset_fn: A callable that takes a client_id and returns a tf.data.Dataset object.

Properties

client_ids

The list of string identifiers for clients in this dataset.

element_type_structure

The element type information of the client datasets.

Returns:

A nested structure of tf.TensorSpec objects defining the type of the

elements returned by datasets in this ClientData object.

Methods

create_from_dir

View source

@classmethod
create_from_dir(
    cls,
    path,
    create_tf_dataset_fn=tf.data.TFRecordDataset
)

Builds a tff.simulation.FilePerUserClientData.

Iterates over all files in path, using the filename as the client ID. Does not recursively search path.

Args:

• path: A directory path to search for per-client files.
• create_tf_dataset_fn: A callable that creates a tf.data.Datasaet object for a given file in the directory specified in path.

Returns:

A tff.simulation.FilePerUserClientData object.

create_tf_dataset_for_client

View source

create_tf_dataset_for_client(client_id)

Creates a new tf.data.Dataset containing the client training examples.

Args:

• client_id: The string client_id for the desired client.

Returns:

A tf.data.Dataset object.

create_tf_dataset_from_all_clients

View source

create_tf_dataset_from_all_clients(seed=None)

Creates a new tf.data.Dataset containing all client examples.

NOTE: the returned tf.data.Dataset is not serializable and runnable on other devices, as it uses tf.py_func internally.

Currently, the implementation produces a dataset that contains all examples from a single client in order, and so generally additional shuffling should be performed.

Args:

• seed: Optional, a seed to determine the order in which clients are processed in the joined dataset.

Returns:

A tf.data.Dataset object.

from_clients_and_fn

View source

from_clients_and_fn(
    cls,
    client_ids,
    create_tf_dataset_for_client_fn
)

Constructs a ClientData based on the given function.

Args:

• client_ids: A non-empty list of client_ids which are valid inputs to the create_tf_dataset_for_client_fn.
• create_tf_dataset_for_client_fn: A function that takes a client_id from the above list, and returns a tf.data.Dataset.

Returns:

A ClientData.

preprocess

View source

preprocess(preprocess_fn)

Applies preprocess_fn to each client's data.

train_test_client_split

View source

train_test_client_split(
    cls,
    client_data,
    num_test_clients
)

Returns a pair of (train, test) ClientData.

This method partitions the clients of client_data into two ClientData objects with disjoint sets of ClientData.client_ids. All clients in the test ClientData are guaranteed to have non-empty datasets, but the training ClientData may have clients with no data.

Args:

• client_data: The base ClientData to split.
• num_test_clients: How many clients to hold out for testing. This can be at most len(client_data.client_ids) - 1, since we don't want to produce empty ClientData.

Returns:

A pair (train_client_data, test_client_data), where test_client_data has num_test_clients selected at random, subject to the constraint they each have at least 1 batch in their dataset.

Raises:

• ValueError: If num_test_clients cannot be satistifed by client_data, or too many clients have empty datasets.