Questions about TFX? Join us at Google I/O!


Task generator for executing an async pipeline.

Inherits From: TaskGenerator

Calling generate is not thread-safe. Concurrent calls to generate should be explicitly serialized. Since MLMD may be updated upon call to generate, it's also not safe to call generate on different instances of this class where the instances refer to the same MLMD db and the same pipeline IR.

mlmd_handle A handle to MLMD db.
pipeline_state Pipeline state.
is_task_id_tracked_fn A callable that returns True if a task_id is tracked by the task queue.
service_job_manager Used for handling service nodes in the pipeline.
ignore_node_ids Set of node ids of nodes to ignore for task generation.



View source

Generates tasks for all executable nodes in the async pipeline.

The returned tasks must have exec_task populated. List may be empty if no nodes are ready for execution.

A list of tasks to execute.