tf.keras.utils.SequenceEnqueuer

TensorFlow 1 version View source on GitHub

Base class to enqueue inputs.

The task of an Enqueuer is to use parallelism to speed up preprocessing. This is done with processes or threads.

Example:

    enqueuer = SequenceEnqueuer(...)
    enqueuer.start()
    datas = enqueuer.get()
    for data in datas:
        # Use the inputs; training, evaluating, predicting.
        # ... stop sometime.
    enqueuer.stop()

The enqueuer.get() should be an infinite stream of datas.

Methods

get

View source

Creates a generator to extract data from the queue.

Skip the data if it is None.

Returns

Generator yielding tuples `(inputs, targets)`
    or `(inputs, targets, sample_weights)`.

is_running

View source

start

View source

Starts the handler's workers.

Arguments
workers Number of workers.
max_queue_size queue size (when full, workers could block on put())

stop

View source

Stops running threads and wait for them to exit, if necessary.

Should be called by the same thread which called start().

Arguments
timeout maximum time to wait on thread.join()