tfm.core.input_reader.InputReader

Input reader that returns a tf.data.Dataset instance.

params A config_definitions.DataConfig object.
dataset_fn A tf.data.Dataset that consumes the input files. For example, it can be tf.data.TFRecordDataset.
decoder_fn An optional callable that takes the serialized data string and decodes them into the raw tensor dictionary.
combine_fn An optional callable that takes a dictionarty of tf.data.Dataset objects as input and outputs a combined dataset. It will be executed after the decoder_fn and before the sample_fn.
sample_fn An optional callable that takes a tf.data.Dataset object as input and outputs the transformed dataset. It performs sampling on the decoded raw tensors dict before the parser_fn.
parser_fn An optional callable that takes the decoded raw tensors dict and parse them into a dictionary of tensors that can be consumed by the model. It will be executed after decoder_fn.
transform_and_batch_fn An optional callable that takes a tf.data.Dataset object and an optional tf.distribute.InputContext as input, and returns a tf.data.Dataset object. It will be executed after parser_fn to transform and batch the dataset; if None, after parser_fn is executed, the dataset will be batched into per-replica batch size.
postprocess_fn A optional callable that processes batched tensors. It will be executed after batching.

tfds_info Returns TFDS dataset info, if available.

Methods

get_files

View source

Gets matched files. Can be overridden by subclasses.

read

View source

Generates a tf.data.Dataset object.

static_randnum 1229806952