Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tfds.file_adapter.TFRecordExampleAdapter

View source on GitHub

Writes/Reads serialized Examples protos to/from TFRecord files.

Inherits From: FileFormatAdapter

tfds.file_adapter.TFRecordExampleAdapter(
    example_specs
)

Constraints on generators:

  • The generator must yield feature dictionaries (dict<str feature_name, feature_value>).
  • The allowed feature types are int, float, and str (or bytes in Python 3; unicode strings will be encoded in utf-8), or lists thereof.

Args:

Attributes:

  • filetype_suffix: Returns a str file type suffix (e.g. "tfrecord").

Methods

dataset_from_filename

View source

dataset_from_filename(
    filename
)

Returns a tf.data.Dataset whose elements are dicts given a filename.

write_from_generator

View source

write_from_generator(
    generator, output_files
)

Write to files from generators_and_filenames.

Args:

  • generator: generator yielding dictionaries of feature name to value.
  • output_files: list<str>, output files to write files to.

write_from_pcollection

View source

write_from_pcollection(
    pcollection, file_path_prefix, num_shards
)

Write the PCollection to file.

Args:

  • pcollection: beam.PCollection, the PCollection containing the examples to write.
  • file_path_prefix: str, output files to write files to.
  • num_shards: int,