tfds.builder_from_directories

Loads a tfds.core.DatasetBuilder from the given generated dataset path.

tfds.builder_from_directories(
    builder_dirs: List[tfds.typing.PathLike],
    *,
    filetype_suffix: Optional[str] = None
) -> tfds.core.DatasetBuilder

When a dataset is spread out over multiple folders, then this function can be used to easily read from all builder dirs.

Note that the data in each folder must have the same features, dataset name, and version.

Some examples of when a dataset might be spread out over multiple folders:

  • in reinforcement learning, multiple agents each produce a dataset
  • each day a new dataset is produced based on new incoming events

builder_dirs the list of builder dirs from which the data should be read.
filetype_suffix DEPRECATED PLEASE DO NOT USE. The filetype suffix (e.g. 'tfrecord') that is used if the file format is not specified in the DatasetInfo.

the read only dataset builder that is configured to read from all the given builder dirs.