Creates a dataset that applies
f to the outputs of
Compat aliases for migration
See Migration guide for more details.
tf.raw_ops.ParallelInterleaveDataset( input_dataset, other_arguments, cycle_length, block_length, sloppy, buffer_output_elements, prefetch_input_elements, f, output_types, output_shapes, name=None )
The resulting dataset is similar to the
InterleaveDataset, with the exception
that if retrieving the next value from a dataset would cause the requester to
block, it will skip that input dataset. This dataset is especially useful
when loading data from a variable-latency datastores (e.g. HDFS, GCS), as it
allows the training step to proceed so long as some data is available.
!! WARNING !! If the
sloppy parameter is set to
True, the operation of this
dataset will not be deterministic!
This dataset has been superseded by
ParallelInterleaveDatasetV2. New code
variant. Dataset that produces a stream of arguments for the function
other_arguments: A list of
Tensorobjects. Additional arguments to pass to
fbeyond those produced by
input_dataset. Evaluated once when the dataset is instantiated.
int64. Number of datasets (each created by applying
fto the elements of
input_dataset) among which the
ParallelInterleaveDatasetwill cycle in a round-robin fashion.
int64. Number of elements at a time to produce from each interleaved invocation of a dataset returned by
True, return elements as they become available, even if that means returning these elements in a non-deterministic order. Sloppy operation may result in better performance in the presence of stragglers, but the dataset will still block if all of its open streams are blocked. If
False, always return elements in a deterministic order.
int64. The number of elements each iterator being interleaved should buffer (similar to the
.prefetch()transformation for each interleaved iterator).
int64. Determines the number of iterators to prefetch, allowing buffers to warm up and data to be pre-fetched without blocking the main thread.
f: A function decorated with @Defun. A function mapping elements of
input_dataset, concatenated with
other_arguments, to a Dataset variant that contains elements matching
output_types: A list of
tf.DTypesthat has length
output_shapes: A list of shapes (each a
tf.TensorShapeor list of
ints) that has length
name: A name for the operation (optional).
Tensor of type