tf.raw_ops.SnapshotDataset

Creates a dataset that will write to / read from a snapshot.

tf.raw_ops.SnapshotDataset(
    input_dataset, path, output_types, output_shapes, compression='',
    reader_path_prefix='', writer_path_prefix='', shard_size_bytes=10737418240,
    pending_snapshot_expiry_seconds=86400, num_reader_threads=1,
    reader_buffer_size=1, num_writer_threads=1, writer_buffer_size=1,
    shuffle_on_read=False, seed=0, seed2=0, mode='auto', snapshot_name='', name=None
)

This dataset attempts to determine whether a valid snapshot exists at the snapshot_path, and reads from the snapshot in lieu of using input_dataset. If not, it will run the preprocessing pipeline as usual, and write out a snapshot of the data processed for future use.

Args:

  • input_dataset: A Tensor of type variant. A variant tensor representing the input dataset.
  • path: A Tensor of type string. The path we should write snapshots to / read snapshots from.
  • output_types: A list of tf.DTypes that has length >= 1.
  • output_shapes: A list of shapes (each a tf.TensorShape or list of ints) that has length >= 1.
  • compression: An optional string. Defaults to "".
  • reader_path_prefix: An optional string. Defaults to "".
  • writer_path_prefix: An optional string. Defaults to "".
  • shard_size_bytes: An optional int. Defaults to 10737418240.
  • pending_snapshot_expiry_seconds: An optional int. Defaults to 86400.
  • num_reader_threads: An optional int. Defaults to 1.
  • reader_buffer_size: An optional int. Defaults to 1.
  • num_writer_threads: An optional int. Defaults to 1.
  • writer_buffer_size: An optional int. Defaults to 1.
  • shuffle_on_read: An optional bool. Defaults to False.
  • seed: An optional int. Defaults to 0.
  • seed2: An optional int. Defaults to 0.
  • mode: An optional string. Defaults to "auto".
  • snapshot_name: An optional string. Defaults to "".
  • name: A name for the operation (optional).

Returns:

A Tensor of type variant.