Questions about TFX? Join us at Google I/O!

tfx.orchestration.experimental.core.task_scheduler.TaskScheduler

Interface for task schedulers.

tfx.orchestration.experimental.core.task_scheduler.TaskScheduler(
    mlmd_handle: tfx.orchestration.metadata.Metadata,
    pipeline: pipeline_pb2.Pipeline,
    task: tfx.orchestration.experimental.core.task.Task
)

mlmd_handle A handle to the MLMD db.
pipeline The pipeline IR proto.
task Task to be executed.

Methods

cancel

View source

@abc.abstractmethod
cancel() -> None

Cancels task scheduler.

This method will be invoked from a different thread than the thread that's blocked on call to schedule. cancel must return immediately when called. Upon cancellation, schedule method is expected to stop any ongoing work, clean up and return as soon as possible. It's technically possible for cancel to be invoked before schedule; scheduler implementations should handle this case by returning from schedule immediately.

schedule

View source

@abc.abstractmethod
schedule() -> tfx.orchestration.experimental.core.task_scheduler.TaskSchedulerResult

Schedules task execution and returns the results of execution.

This method blocks until task execution completes (successfully or not) or until explicitly cancelled by a call to cancel. When cancelled, schedule is expected to stop any ongoing work, clean up and return as soon as possible. Note that cancel will be invoked from a different thread than schedule and hence the concrete implementations must be thread safe. It's technically possible for cancel to be invoked before schedule; scheduler implementations should handle this case by returning from schedule immediately.