tf.data.experimental.service.from_dataset_id

Creates a dataset which reads data from the tf.data service.

This is useful when the dataset is registered by one process, then used in another process. When the same process is both registering and reading from the dataset, it is simpler to use tf.data.experimental.service.distribute instead.

Before using from_dataset_id, the dataset must have been registered with the tf.data service using tf.data.experimental.service.register_dataset. register_dataset returns a dataset id for the registered dataset. That is the dataset_id which should be passed to from_dataset_id.

The element_spec argument indicates the tf.TypeSpecs for the elements produced by the dataset. Currently element_spec must be explicitly specified, and match the dataset registered under dataset_id. element_spec defaults to None so that in the future we can support automatically discovering the element_spec by querying the tf.data service.

tf.data.experimental.service.distribute is a convenience method which combines register_dataset and from_dataset_id into a dataset transformation. See the documentation for tf.data.experimental.service.distribute for more detail about how from_dataset_id works.

dispatcher = tf.data.experimental.service.DispatchServer()
dispatcher_address = dispatcher.target.split("://")[1]
worker = tf.data.experimental.service.WorkerServer(
    tf.data.exper