Save the date! Google I/O returns May 18-20 Register now

tfds.download.DownloadConfig

Configuration for tfds.core.DatasetBuilder.download_and_prepare.

tfds.download.DownloadConfig(
    extract_dir: Optional[utils.PathLike] = None,
    manual_dir: Optional[utils.PathLike] = None,
    download_mode: tfds.download.GenerateMode = tfds.download.DownloadConfig.download_mode,
    compute_stats: tfds.download.ComputeStatsMode = tfds.download.ComputeStatsMode.SKIP,
    max_examples_per_split: tfds.typing.Dim = None,
    register_checksums: bool = False,
    force_checksums_validation: bool = False,
    beam_runner: Optional[Any] = None,
    beam_options: Optional[Any] = None,
    try_download_gcs: bool = True,
    verify_ssl: bool = True
)

extract_dir str, directory where extracted files are stored. Defaults to "/extracted".
manual_dir str, read-only directory where manually downloaded/extracted data is stored. Defaults to <download_dir>/manual.
download_mode tfds.GenerateMode, how to deal with downloads or data that already exists. Defaults to REUSE_DATASET_IF_EXISTS, which will reuse both downloads and data if it already exists.
compute_stats tfds.download.ComputeStats, whether to compute statistics over the generated data. Defaults to AUTO.
max_examples_per_split int, optional max number of examples to write into each split (used for testing). If set to 0, only execute the _split_generators (download the original data), but skip _generator_examples.
register_checksums bool, defaults to False. If True, checksum of downloaded files are recorded.
force_checksums_validation bool, defaults to False. If True, raises an error if an URL do not have checksums.
beam_runner Runner to pass to beam.Pipeline, only used for datasets based on Beam for the generation.
beam_options PipelineOptions to pass to beam.Pipeline, only used for datasets based on Beam for the generation.
try_download_gcs bool, defaults to True. If True, prepared dataset will be downloaded from GCS, when available. If False, dataset will be downloaded and prepared from scratch.
verify_ssl bool, defaults to True. If True, will verify certificate when downloading dataset.

beam_options None
beam_runner None
compute_stats <ComputeStatsMode.SKIP: 'skip'>
download_mode <GenerateMode.REUSE_DATASET_IF_EXISTS: 'reuse_dataset_if_exists'>
extract_dir None
force_checksums_validation False
manual_dir None
max_examples_per_split None
register_checksums False
try_download_gcs True
verify_ssl True