|View source on GitHub|
Holds a list of enqueue operations for a queue, each to be run in a thread.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.QueueRunner( queue=None, enqueue_ops=None, close_op=None, cancel_op=None, queue_closed_exception_types=None, queue_runner_def=None, import_scope=None )
Queues are a convenient TensorFlow mechanism to compute tensors asynchronously using multiple threads. For example in the canonical 'Input Reader' setup one set of threads generates filenames in a queue; a second set of threads read records from the files, processes them, and enqueues tensors on a second queue; a third set of threads dequeues these input records to construct batches and runs them through training operations.
There are several delicate issues when running multiple threads that way: closing the queues in sequence as the input is exhausted, correctly catching and reporting exceptions, etc.
QueueRunner, combined with the
Coordinator, helps handle these issues.
enqueue_ops: List of enqueue ops to run in threads later.
close_op: Op to close the queue. Pending enqueue ops are preserved.
cancel_op: Op to close the queue and cancel pending enqueue ops.
queue_closed_exception_types: Optional tuple of Exception types that indicate that the queue has been closed when raised during an enqueue operation. Defaults to
(tf.errors.OutOfRangeError,). Another common case includes
(tf.errors.OutOfRangeError, tf.errors.CancelledError), when some of the enqueue ops may dequeue from other Queues.
QueueRunnerDefprotocol buffer. If specified, recreates the QueueRunner from its contents.
queue_runner_defand the other arguments are mutually exclusive.
string. Name scope to add. Only used when initializing from protocol buffer.
exceptions_raised: Exceptions raised but not handled by the
Exceptions raised in queue runner threads are handled in one of two ways depending on whether or not a
Coordinatorwas passed to
- With a
Coordinator, exceptions are reported to the coordinator and forgotten by the
- Without a
Coordinator, exceptions are captured by the
QueueRunnerand made available in this
- With a
name: The string name of the underlying Queue.
ValueError: If both
queueare both specified.
enqueue_opsare not provided when not restoring from
RuntimeError: If eager execution is enabled.
QueueRunners are not compatible with eager execution. Instead, please
tf.data to get data into your model.
create_threads( sess, coord=None, daemon=False, start=False )
Create threads to run the enqueue ops for the given session.
This method requires a session in which the graph was launched. It creates
a list of threads, optionally starting them. There is one thread for each
op passed in
coord argument is an optional coordinator that the threads will use
to terminate together and report exceptions. If a coordinator is given,
this method starts an additional thread to close the queue when the
coordinator requests a stop.
If previously created threads for the given session are still running, no new threads will be created.
Coordinatorobject for reporting errors and checking stop conditions.
daemon: Boolean. If
Truemake the threads daemon threads.
start: Boolean. If
Truestarts the threads. If
Falsethe caller must call the
start()method of the returned threads.
A list of threads.
@staticmethod from_proto( queue_runner_def, import_scope=None )
QueueRunner object created from
to_proto( export_scope=None )
QueueRunner to a
QueueRunnerDef protocol buffer.
string. Name scope to remove.
QueueRunnerDef protocol buffer, or
None if the
Variable is not in
the specified name scope.