Creates a dataset that will write to / read from a snapshot.
tf.raw_ops.SnapshotDataset(
input_dataset,
path,
output_types,
output_shapes,
compression='',
reader_path_prefix='',
writer_path_prefix='',
shard_size_bytes=10737418240,
pending_snapshot_expiry_seconds=86400,
num_reader_threads=1,
reader_buffer_size=1,
num_writer_threads=1,
writer_buffer_size=1,
shuffle_on_read=False,
seed=0,
seed2=0,
mode='auto',
snapshot_name='',
name=None
)
This dataset attempts to determine whether a valid snapshot exists at the
snapshot_path
, and reads from the snapshot in lieu of using input_dataset
.
If not, it will run the preprocessing pipeline as usual, and write out a
snapshot of the data processed for future use.
Args | |
---|---|
input_dataset
|
A Tensor of type variant .
A variant tensor representing the input dataset.
|
path
|
A Tensor of type string .
The path we should write snapshots to / read snapshots from.
|
output_types
|
A list of tf.DTypes that has length >= 1 .
|
output_shapes
|
A list of shapes (each a tf.TensorShape or list of ints ) that has length >= 1 .
|
compression
|
An optional string . Defaults to "" .
|
reader_path_prefix
|
An optional string . Defaults to "" .
|
writer_path_prefix
|
An optional string . Defaults to "" .
|
shard_size_bytes
|
An optional int . Defaults to 10737418240 .
|
pending_snapshot_expiry_seconds
|
An optional int . Defaults to 86400 .
|
num_reader_threads
|
An optional int . Defaults to 1 .
|
reader_buffer_size
|
An optional int . Defaults to 1 .
|
num_writer_threads
|
An optional int . Defaults to 1 .
|
writer_buffer_size
|
An optional int . Defaults to 1 .
|
shuffle_on_read
|
An optional bool . Defaults to False .
|
seed
|
An optional int . Defaults to 0 .
|
seed2
|
An optional int . Defaults to 0 .
|
mode
|
An optional string . Defaults to "auto" .
|
snapshot_name
|
An optional string . Defaults to "" .
|
name
|
A name for the operation (optional). |
Returns | |
---|---|
A Tensor of type variant .
|