Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tfds.file_adapter.FileFormatAdapter

View source on GitHub

Provides writing and reading methods for a file format.

tfds.file_adapter.FileFormatAdapter(
    example_specs
)

Args:

Attributes:

  • filetype_suffix: Returns a str file type suffix (e.g. "tfrecord").

Methods

dataset_from_filename

View source

dataset_from_filename(
    filename
)

Returns a tf.data.Dataset whose elements are dicts given a filename.

write_from_generator

View source

write_from_generator(
    generator, output_files
)

Write to files from generators_and_filenames.

Args:

  • generator: generator yielding dictionaries of feature name to value.
  • output_files: list<str>, output files to write files to.

write_from_pcollection

View source

write_from_pcollection(
    pcollection, file_path_prefix=None, num_shards=None
)

Write the PCollection to file.

Args:

  • pcollection: beam.PCollection, the PCollection containing the examples to write.
  • file_path_prefix: str, output files to write files to.
  • num_shards: int,