Configuration for tfds.core.DatasetBuilder.download_and_prepare
.
tfds.download.DownloadConfig(
extract_dir: Optional[epath.PathLike] = None,
manual_dir: Optional[epath.PathLike] = None,
download_mode: tfds.download.GenerateMode
= tfds.download.DownloadConfig.download_mode
,
compute_stats: tfds.download.ComputeStatsMode
= tfds.download.ComputeStatsMode.SKIP
,
max_examples_per_split: tfds.typing.Dim
= None,
register_checksums: bool = False,
force_checksums_validation: bool = False,
beam_runner: Optional[Any] = None,
beam_options: Optional[Any] = None,
try_download_gcs: bool = True,
verify_ssl: bool = True
)
Attributes |
extract_dir
|
str , directory where extracted files are stored. Defaults to
"/extracted".
|
manual_dir
|
str , read-only directory where manually downloaded/extracted
data is stored. Defaults to <download_dir>/manual .
|
download_mode
|
tfds.GenerateMode , how to deal with downloads or data that
already exists. Defaults to REUSE_DATASET_IF_EXISTS , which will reuse
both downloads and data if it already exists.
|
compute_stats
|
tfds.download.ComputeStats , whether to compute statistics
over the generated data. Defaults to AUTO .
|
max_examples_per_split
|
int , optional max number of examples to write into
each split (used for testing). If set to 0, only execute the
_split_generators (download the original data), but skip
_generator_examples .
|
register_checksums
|
bool , defaults to False. If True, checksum of
downloaded files are recorded.
|
force_checksums_validation
|
bool , defaults to False. If True, raises an
error if an URL do not have checksums.
|
beam_runner
|
Runner to pass to beam.Pipeline , only used for datasets based
on Beam for the generation.
|
beam_options
|
PipelineOptions to pass to beam.Pipeline , only used for
datasets based on Beam for the generation.
|
try_download_gcs
|
bool , defaults to True. If True, prepared dataset will
be downloaded from GCS, when available. If False, dataset will be
downloaded and prepared from scratch.
|
verify_ssl
|
bool , defaults to True. If True, will verify certificate when
downloading dataset.
|
Class Variables |
beam_options
|
None
|
beam_runner
|
None
|
compute_stats
|
<ComputeStatsMode.SKIP: 'skip'>
|
download_mode
|
<GenerateMode.REUSE_DATASET_IF_EXISTS: 'reuse_dataset_if_exists'>
|
extract_dir
|
None
|
force_checksums_validation
|
False
|
manual_dir
|
None
|
max_examples_per_split
|
None
|
register_checksums
|
False
|
try_download_gcs
|
True
|
verify_ssl
|
True
|