Readers

TensorFlow provides a set of Reader classes for reading data formats. For more information on inputs and readers, see Reading data.

class tf.ReaderBase

Base class for different Reader types, that produce a record every step.

Conceptually, Readers convert string 'work units' into records (key, value pairs). Typically the 'work units' are filenames and the records are extracted from the contents of those files. We want a single record produced per step, but a work unit can correspond to many records.

Therefore we introduce some decoupling using a queue. The queue contains the work units and the Reader dequeues from the queue when it is asked to produce a record (via Read()) but it has finished the last work unit.


tf.ReaderBase.__init__(reader_ref, supports_serialize=False) {:#ReaderBase.init}

Creates a new ReaderBase.

Args:
  • reader_ref: The operation that implements the reader.
  • supports_serialize: True if the reader implementation can serialize its state.

tf.ReaderBase.num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.ReaderBase.num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.ReaderBase.read(queue, name=None)

Returns the next record (key, value pair) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (key, value).

  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

tf.ReaderBase.read_up_to(queue, num_records, name=None)

Returns up to num_records (key, value pairs) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (keys, values).

  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

tf.ReaderBase.reader_ref

Op that implements the reader.


tf.ReaderBase.reset(name=None)

Restore a reader to its initial clean state.

Args:
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.ReaderBase.restore_state(state, name=None)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:
  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.ReaderBase.serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:
  • name: A name for the operation (optional).
Returns:

A string Tensor.


tf.ReaderBase.supports_serialize

Whether the Reader implementation can serialize its state.


class tf.TextLineReader

A Reader that outputs the lines of a file delimited by newlines.

Newlines are stripped from the output. See ReaderBase for supported methods.


tf.TextLineReader.__init__(skip_header_lines=None, name=None) {:#TextLineReader.init}

Create a TextLineReader.

Args:
  • skip_header_lines: An optional int. Defaults to 0. Number of lines to skip from the beginning of every file.
  • name: A name for the operation (optional).

tf.TextLineReader.num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.TextLineReader.num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.TextLineReader.read(queue, name=None)

Returns the next record (key, value pair) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (key, value).

  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

tf.TextLineReader.read_up_to(queue, num_records, name=None)

Returns up to num_records (key, value pairs) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (keys, values).

  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

tf.TextLineReader.reader_ref

Op that implements the reader.


tf.TextLineReader.reset(name=None)

Restore a reader to its initial clean state.

Args:
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.TextLineReader.restore_state(state, name=None)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:
  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.TextLineReader.serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:
  • name: A name for the operation (optional).
Returns:

A string Tensor.


tf.TextLineReader.supports_serialize

Whether the Reader implementation can serialize its state.


class tf.WholeFileReader

A Reader that outputs the entire contents of a file as a value.

To use, enqueue filenames in a Queue. The output of Read will be a filename (key) and the contents of that file (value).

See ReaderBase for supported methods.


tf.WholeFileReader.__init__(name=None) {:#WholeFileReader.init}

Create a WholeFileReader.

Args:
  • name: A name for the operation (optional).

tf.WholeFileReader.num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.WholeFileReader.num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.WholeFileReader.read(queue, name=None)

Returns the next record (key, value pair) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (key, value).

  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

tf.WholeFileReader.read_up_to(queue, num_records, name=None)

Returns up to num_records (key, value pairs) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (keys, values).

  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

tf.WholeFileReader.reader_ref

Op that implements the reader.


tf.WholeFileReader.reset(name=None)

Restore a reader to its initial clean state.

Args:
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.WholeFileReader.restore_state(state, name=None)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:
  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.WholeFileReader.serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:
  • name: A name for the operation (optional).
Returns:

A string Tensor.


tf.WholeFileReader.supports_serialize

Whether the Reader implementation can serialize its state.


class tf.IdentityReader

A Reader that outputs the queued work as both the key and value.

To use, enqueue strings in a Queue. Read will take the front work string and output (work, work).

See ReaderBase for supported methods.


tf.IdentityReader.__init__(name=None) {:#IdentityReader.init}

Create a IdentityReader.

Args:
  • name: A name for the operation (optional).

tf.IdentityReader.num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.IdentityReader.num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.IdentityReader.read(queue, name=None)

Returns the next record (key, value pair) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (key, value).

  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

tf.IdentityReader.read_up_to(queue, num_records, name=None)

Returns up to num_records (key, value pairs) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (keys, values).

  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

tf.IdentityReader.reader_ref

Op that implements the reader.


tf.IdentityReader.reset(name=None)

Restore a reader to its initial clean state.

Args:
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.IdentityReader.restore_state(state, name=None)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:
  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.IdentityReader.serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:
  • name: A name for the operation (optional).
Returns:

A string Tensor.


tf.IdentityReader.supports_serialize

Whether the Reader implementation can serialize its state.


class tf.TFRecordReader

A Reader that outputs the records from a TFRecords file.

See ReaderBase for supported methods.


tf.TFRecordReader.__init__(name=None, options=None) {:#TFRecordReader.init}

Create a TFRecordReader.

Args:
  • name: A name for the operation (optional).
  • options: A TFRecordOptions object (optional).

tf.TFRecordReader.num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.TFRecordReader.num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.TFRecordReader.read(queue, name=None)

Returns the next record (key, value pair) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (key, value).

  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

tf.TFRecordReader.read_up_to(queue, num_records, name=None)

Returns up to num_records (key, value pairs) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (keys, values).

  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

tf.TFRecordReader.reader_ref

Op that implements the reader.


tf.TFRecordReader.reset(name=None)

Restore a reader to its initial clean state.

Args:
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.TFRecordReader.restore_state(state, name=None)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:
  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.TFRecordReader.serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:
  • name: A name for the operation (optional).
Returns:

A string Tensor.


tf.TFRecordReader.supports_serialize

Whether the Reader implementation can serialize its state.


class tf.FixedLengthRecordReader

A Reader that outputs fixed-length records from a file.

See ReaderBase for supported methods.


Create a FixedLengthRecordReader.

Args:
  • record_bytes: An int.
  • header_bytes: An optional int. Defaults to 0.
  • footer_bytes: An optional int. Defaults to 0.
  • name: A name for the operation (optional).

tf.FixedLengthRecordReader.num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.FixedLengthRecordReader.num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:
  • name: A name for the operation (optional).
Returns:

An int64 Tensor.


tf.FixedLengthRecordReader.read(queue, name=None)

Returns the next record (key, value pair) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (key, value).

  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

tf.FixedLengthRecordReader.read_up_to(queue, num_records, name=None)

Returns up to num_records (key, value pairs) produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:
  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).
Returns:

A tuple of Tensors (keys, values).

  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

tf.FixedLengthRecordReader.reader_ref

Op that implements the reader.


tf.FixedLengthRecordReader.reset(name=None)

Restore a reader to its initial clean state.

Args:
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.FixedLengthRecordReader.restore_state(state, name=None)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:
  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).
Returns:

The created Operation.


tf.FixedLengthRecordReader.serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:
  • name: A name for the operation (optional).
Returns:

A string Tensor.


tf.FixedLengthRecordReader.supports_serialize

Whether the Reader implementation can serialize its state.