|---| | [ View source on GitHub](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builders/adhoc_builder.py#L260-L276) | Builds datasets from tf.data.Dataset without defining a class. Inherits From: [`AdhocBuilder`](https://www.tensorflow.org/datasets/api_docs/python/tfds/dataset_builders/AdhocBuilder), [`GeneratorBasedBuilder`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/GeneratorBasedBuilder), [`DatasetBuilder`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetBuilder) tfds.dataset_builders.TfDataBuilder( *, split_datasets: Mapping[str, tf.data.Dataset], **kwargs ) This is kept for backwards-compatibility.


| ## Args || |---|---| | `file_format` | EXPERIMENTAL, may change at any time; Format of the record files in which dataset will be read/written to. If `None`, defaults to `tfrecord`. | | `**kwargs` | Arguments passed to `DatasetBuilder`. |



| ## Attributes || |---|---| | `blocked_versions` |

| | `builder_config` | [`tfds.core.BuilderConfig`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/BuilderConfig) for this builder. | | `builder_config_name` | Name of the [`tfds.core.BuilderConfig`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/BuilderConfig) for this builder. | | `canonical_version` |

| | `data_dir` | Returns the directory where this version + config is stored.
Note that this is different from `data_dir_root`. For example, if `data_dir_root` is `/data/tfds`, then `data_dir` would be `/data/tfds/my_dataset/my_config/1.2.3`. | | `data_dir_root` | Returns the root directory where all TFDS datasets are stored.
Note that this is different from `data_dir`, which includes the dataset name, config, and version. For example, if `data_dir` is `/data/tfds/my_dataset/my_config/1.2.3`, then `data_dir_root` is `/data/tfds`. | | `data_path` | Returns the path where this version + config is stored. | | `info` | [`tfds.core.DatasetInfo`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetInfo) for this builder. | | `release_notes` |

| | `supported_versions` |

| | `version` |

| | `versions` | Versions (canonical + availables), in preference order. |
## Methods ### `as_data_source` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L821-L942) as_data_source( split: (Tree[splits_lib.SplitArg] | None) = None, *, decoders: (TreeDict[decode.partial_decode.DecoderArg] | None) = None, deserialize_method: decode.DeserializeMethod = https://www.tensorflow.org/datasets/api_docs/python/tfds/decode/DeserializeMethod#DESERIALIZE_AND_DECODE, file_format: (str | file_adapters.FileFormat | None) = None ) -> ListOrTreeOrElem[Sequence[Any]] Constructs an `ArrayRecordDataSource`.


| Args || |---|---| | `split` | Which split of the data to load (e.g. `'train'`, `'test'`, `['train', 'test']`, `'train[80%:]'`,...). See our [split API guide](https://www.tensorflow.org/datasets/splits). If `None`, will return all splits in a `dict[Split, Sequence]`. | | `decoders` | Nested dict of `Decoder` objects which allow to customize the decoding. The structure should match the feature structure, but only customized feature keys need to be present. See [the guide](https://github.com/tensorflow/datasets/blob/master/docs/decode.md) for more info. | | `deserialize_method` | Whether the read examples should be deserialized and/or decoded. If not specified, it'll deserialize the data and decode the features. Decoding is only supported if the examples are tf examples. Note that if the deserialize_method method is other than PARSE_AND_DECODE, then the `decoders` argument is ignored. | | `file_format` | if the dataset is stored in multiple file formats, then this can be used to specify which format to use. If not provided, we will default to the first available format. |



| Returns || |---|---| | `Sequence` if `split`, `dict` otherwise. ||



| Raises || |---|---| | NotImplementedError if the data was not generated using ArrayRecords. ||
### `as_dataset` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L944-L1058) as_dataset( split: (Tree[splits_lib.SplitArg] | None) = None, *, batch_size: (int | None) = None, shuffle_files: bool = False, decoders: (TreeDict[decode.partial_decode.DecoderArg] | None) = None, read_config: (read_config_lib.ReadConfig | None) = None, as_supervised: bool = False ) Constructs a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). Callers must pass arguments as keyword arguments. The output types vary depending on the parameters. Examples: builder = tfds.builder('imdb_reviews') builder.download_and_prepare() # Default parameters: Returns the dict of tf.data.Dataset ds_all_dict = builder.as_dataset() assert isinstance(ds_all_dict, dict) print(ds_all_dict.keys()) # ==> ['test', 'train', 'unsupervised'] assert isinstance(ds_all_dict['test'], tf.data.Dataset) # Each dataset (test, train, unsup.) consists of dictionaries # {'label': , # 'text': } # {'label': , # 'text': } # With as_supervised: tf.data.Dataset only contains (feature, label) tuples ds_all_supervised = builder.as_dataset(as_supervised=True) assert isinstance(ds_all_supervised, dict) print(ds_all_supervised.keys()) # ==> ['test', 'train', 'unsupervised'] assert isinstance(ds_all_supervised['test'], tf.data.Dataset) # Each dataset (test, train, unsup.) consists of tuples (text, label) # (, # ) # (, # ) # Same as above plus requesting a particular split ds_test_supervised = builder.as_dataset(as_supervised=True, split='test') assert isinstance(ds_test_supervised, tf.data.Dataset) # The dataset consists of tuples (text, label) # (, # ) # (, # )


| Args || |---|---| | `split` | Which split of the data to load (e.g. `'train'`, `'test'`, `['train', 'test']`, `'train[80%:]'`,...). See our [split API guide](https://www.tensorflow.org/datasets/splits). If `None`, will return all splits in a `Dict[Split, tf.data.Dataset]`. | | `batch_size` | `int`, batch size. Note that variable-length features will be 0-padded if `batch_size` is set. Users that want more custom behavior should use `batch_size=None` and use the [`tf.data`](https://www.tensorflow.org/api_docs/python/tf/data) API to construct a custom pipeline. If `batch_size == -1`, will return feature dictionaries of the whole dataset with [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor)s instead of a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). | | `shuffle_files` | `bool`, whether to shuffle the input files. Defaults to `False`. | | `decoders` | Nested dict of `Decoder` objects which allow to customize the decoding. The structure should match the feature structure, but only customized feature keys need to be present. See [the guide](https://github.com/tensorflow/datasets/blob/master/docs/decode.md) for more info. | | `read_config` | [`tfds.ReadConfig`](https://www.tensorflow.org/datasets/api_docs/python/tfds/ReadConfig), Additional options to configure the input pipeline (e.g. seed, num parallel reads,...). | | `as_supervised` | `bool`, if `True`, the returned [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) will have a 2-tuple structure `(input, label)` according to `builder.info.supervised_keys`. If `False`, the default, the returned [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) will have a dictionary with all the features. |



| Returns || |---|---| | [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset), or if `split=None`, `dict`.
If `batch_size` is -1, will return feature dictionaries containing the entire dataset in [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor)s instead of a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). ||
### `assert_is_not_blocked` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L593-L600) assert_is_not_blocked() -> None Checks that the dataset is not blocked. ### `dataset_info_from_configs` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L1207-L1224) dataset_info_from_configs( **kwargs ) Returns the DatasetInfo using given kwargs and config files. Sub-class should call this and add information not present in config files using kwargs directly passed to tfds.core.DatasetInfo object. If information is present both in passed arguments and config files, config files will prevail.


| Args || |---|---| | `**kwargs` | kw args to pass to DatasetInfo directly. |
### `download_and_prepare` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L602-L801) download_and_prepare( *, download_dir: (epath.PathLike | None) = None, download_config: (download.DownloadConfig | None) = None, file_format: (str | file_adapters.FileFormat | None) = None, permissions: file_utils.Permissions = https://www.tensorflow.org/datasets/api_docs/python/tfds/core/Permissions ) -> None Downloads and prepares dataset for reading.


| Args || |---|---| | `download_dir` | directory where downloaded files are stored. Defaults to "\~/tensorflow-datasets/downloads". | | `download_config` | [`tfds.download.DownloadConfig`](https://www.tensorflow.org/datasets/api_docs/python/tfds/download/DownloadConfig), further configuration for downloading and preparing dataset. | | `file_format` | optional `str` or `file_adapters.FileFormat`, format of the record files in which the dataset will be written. | | `permissions` | permissions to set on the generated folder and files. Defaults to 0o775 instead of gFile's default 0o750. |



| Raises || |---|---| | `IOError` | if there is not enough disk space available. | | `RuntimeError` | when the config cannot be found. | | `DatasetBlockedError` | if the given version, or combination of version and config, has been marked as blocked in the builder's BLOCKED_VERSIONS. |
### `get_builder_config` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L1458-L1469) @classmethod get_builder_config( name: str, version: (str | utils.Version | None) = None ) -> (BuilderConfig | None) Returns the builder config with the given name and version. ### `get_default_builder_config` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L532-L545) get_default_builder_config() -> (BuilderConfig | None) Returns the default builder config if there is one. Note that for dataset builders that cannot use the `cls.BUILDER_CONFIGS`, we need a method that uses the instance to get `BUILDER_CONFIGS` and `DEFAULT_BUILDER_CONFIG_NAME`.


| Returns || |---|---| | the default builder config if there is one ||
### `get_file_spec` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L573-L576) get_file_spec( split: str ) -> (str | None) Returns the file spec of the split. ### `get_metadata` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L250-L257) @classmethod get_metadata() -> dataset_metadata.DatasetMetadata Returns metadata (README, CITATIONS, ...) specified in config files. The config files are read from the same package where the DatasetBuilder has been defined, so those metadata might be wrong for legacy builders. ### `get_reference` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L547-L571) get_reference( namespace: (str | None) = None ) -> naming.DatasetReference Returns a reference to the dataset produced by this dataset builder. Includes the config if specified, the version, and the data_dir that should contain this dataset.


| Arguments || |---|---| | `namespace` | if this dataset is a community dataset, and therefore has a namespace, then the namespace must be provided such that it can be set in the reference. Note that a dataset builder is not aware that it is part of a namespace. |



| Returns || |---|---| | a reference to this instantiated builder. ||
### `is_blocked` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L585-L591) is_blocked() -> utils.IsBlocked Returns whether this builder (version, config) is blocked. ### `is_prepared` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L578-L583) is_prepared() -> bool Returns whether this dataset is already downloaded and prepared. ### `read_text_file` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L1825-L1831) read_text_file( filename: epath.PathLike, encoding: (str | None) = None ) -> str Returns the text in the given file and records the lineage. ### `read_tfrecord_as_dataset` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L1833-L1854) read_tfrecord_as_dataset( filenames: (str | Sequence[str]), compression_type: (str | None) = None, num_parallel_reads: (int | None) = None ) -> tf.data.Dataset Returns the dataset for the given tfrecord files and records the lineage. ### `read_tfrecord_as_examples` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L1856-L1871) read_tfrecord_as_examples( filenames: (str | Sequence[str]), compression_type: (str | None) = None, num_parallel_reads: (int | None) = None ) -> Iterator[tf.train.Example] Returns tf.Examples from the given tfrecord files and records the lineage. ### `read_tfrecord_beam` [View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/dataset_builder.py#L1873-L1897) read_tfrecord_beam( file_pattern, /, **kwargs ) -> beam.PTransform Returns a PTransform reading the TFRecords and records it in the dataset lineage. This function records the lineage in the DatasetInfo and then invokes `beam.io.ReadFromTFRecord`. The kwargs should contain any other parameters for `beam.io.ReadFromTFRecord`. See


| Arguments || |---|---| | `file_pattern` | A file glob pattern to read TFRecords from. | | `**kwargs` | the other parameters for `beam.io.ReadFromTFRecord`. |



| Returns || |---|---| | a Beam PTransform that reads the given TFRecord files. ||



| ## Class Variables || |---|---| | BLOCKED_VERSIONS | `None` | | BUILDER_CONFIGS | `[]` | | DEFAULT_BUILDER_CONFIG_NAME | `None` | | MANUAL_DOWNLOAD_INSTRUCTIONS | `None` | | MAX_SIMULTANEOUS_DOWNLOADS | `None` | | RELEASE_NOTES |
}
| | SUPPORTED_VERSIONS | `[]` | | VERSION | `None` | | builder_config_cls | `None` | | builder_configs |
}
| | code_path | Instance of `etils.epath.gpath.PosixGPath` | | default_builder_config | `None` | | name | `'tf_data_builder'` | | pkg_dir_path | `None` | | url_infos | `None` |