View source on GitHub |
Experimental API for building input pipelines.
This module contains experimental Dataset
sources and transformations that can
be used in conjunction with the tf.data.Dataset
API. Note that the
tf.contrib.data
API is not subject to the same backwards compatibility
guarantees as tf.data
, but we will provide deprecation advice in advance of
removing existing functionality.
See Importing Data for an overview.
Classes
class CheckpointInputPipelineHook
: Checkpoints input pipeline state every N steps or seconds.
class CsvDataset
: A Dataset comprising lines from one or more CSV files.
class LMDBDataset
: A LMDB Dataset that reads the lmdb file.
class Optional
: Wraps a value that may/may not be present at runtime.
class RandomDataset
: A Dataset
of pseudorandom values.
class Reducer
: A reducer is used for reducing a set of elements.
class SqlDataset
: A Dataset
consisting of the results from a SQL query.
class TFRecordWriter
: Writes data to a TFRecord file.
Functions
Counter(...)
: Creates a Dataset
that counts from start
in steps of size step
. (deprecated)
assert_element_shape(...)
: Assert the shape of this Dataset
.
batch_and_drop_remainder(...)
: A batching transformation that omits the final small batch (if present). (deprecated)
bucket_by_sequence_length(...)
: A transformation that buckets elements in a Dataset
by length. (deprecated)
choose_from_datasets(...)
: Creates a dataset that deterministically chooses elements from datasets
. (deprecated)
copy_to_device(...)
: A transformation that copies dataset elements to the given target_device
. (deprecated)
dense_to_sparse_batch(...)
: A transformation that batches ragged elements into tf.SparseTensor
s. (deprecated)
enumerate_dataset(...)
: A transformation that enumerate the elements of a dataset. (deprecated)
get_next_as_optional(...)
: Returns an Optional
that contains the next value from the iterator.
get_single_element(...)
: Returns the single element in dataset
as a nested structure of tensors. (deprecated)
group_by_reducer(...)
: A transformation that groups elements and performs a reduction. (deprecated)
group_by_window(...)
: A transformation that groups windows of elements by key and reduces them. (deprecated)
ignore_errors(...)
: Creates a Dataset
from another Dataset
and silently ignores any errors. (deprecated)
make_batched_features_dataset(...)
: Returns a Dataset
of feature dictionaries from Example
protos. (deprecated)
make_csv_dataset(...)
: Reads CSV files into a dataset. (deprecated)
make_saveable_from_iterator(...)
: Returns a SaveableObject for saving/restore iterator state using Saver. (deprecated)
map_and_batch(...)
: Fused implementation of map
and batch
. (deprecated)
padded_batch_and_drop_remainder(...)
: A batching and padding transformation that omits the final small batch. (deprecated)
parallel_interleave(...)
: A parallel version of the Dataset.interleave()
transformation. (deprecated)
parse_example_dataset(...)
: A transformation that parses Example
protos into a dict
of tensors. (deprecated)
prefetch_to_device(...)
: A transformation that prefetches dataset values to the given device
. (deprecated)
read_batch_features(...)
: Reads batches of Examples. (deprecated)
reduce_dataset(...)
: Returns the result of reducing the dataset
using reducer
. (deprecated)
rejection_resample(...)
: A transformation that resamples a dataset to achieve a target distribution. (deprecated)
sample_from_datasets(...)
: Samples elements at random from the datasets in datasets
. (deprecated)
scan(...)
: A transformation that scans a function across an input dataset. (deprecated)
shuffle_and_repeat(...)
: Shuffles and repeats a Dataset returning a new permutation for each epoch. (deprecated)
sliding_window_batch(...)
: A sliding window over a dataset. (deprecated) (deprecated arguments)
sloppy_interleave(...)
: A non-deterministic version of the Dataset.interleave()
transformation. (deprecated)
unbatch(...)
: Splits elements of a dataset into multiple elements on the batch dimension. (deprecated)
unique(...)
: Creates a Dataset
from another Dataset
, discarding duplicates. (deprecated)