tfx.orchestration.experimental.core.service_jobs.ServiceJobManager

Interface for service job manager.

Service jobs are long-running jobs associated with a node or a pipeline that persist across executions (eg: worker pools, Tensorboard, etc). Service jobs should be started before the nodes that depend on them can be run.

Methods

`ensure_node_services`

View source

@abc.abstractmethod
ensure_node_services(
    pipeline_state: tfx.orchestration.experimental.core.pipeline_state.PipelineState,
    node_id: str
) -> tfx.orchestration.experimental.core.service_jobs.ServiceStatus

Ensures necessary service jobs are started and healthy for the node.

ensure_node_services will be called in the orchestration loop periodically and is expected to:

Start any service jobs required by the pipeline node.
Probe job health, handle failure and return appropriate status.

Note that this method will only be called if either is_pure_service_node or is_mixed_service_node return True for the node.

Args
`pipeline_state`	A `PipelineState` object for an active pipeline.
`node_id`	Id of the node to ensure services.

Returns
Status of the service job(s) for the node.

`is_mixed_service_node`

View source

@abc.abstractmethod
is_mixed_service_node(
    pipeline_state: tfx.orchestration.experimental.core.pipeline_state.PipelineState,
    node_id: str
) -> bool

Returns True if the given node has a mix of executor and service jobs.

Args
`pipeline_state`	A `PipelineState` object for an active pipeline.
`node_id`	Id of the node in the pipeline to be checked.

Returns
`True` if the node has a mix of executor and service jobs.

`is_pure_service_node`

View source

@abc.abstractmethod
is_pure_service_node(
    pipeline_state: tfx.orchestration.experimental.core.pipeline_state.PipelineState,
    node_id: str
) -> bool

Returns True if the given node only has service job(s).

Args
`pipeline_state`	A `PipelineState` object for an active pipeline.
`node_id`	Id of the node in the pipeline to be checked.

Returns
`True` if the node only has service job(s).

`stop_node_services`

View source

@abc.abstractmethod
stop_node_services(
    pipeline_state: tfx.orchestration.experimental.core.pipeline_state.PipelineState,
    node_id: str
) -> None

Stops service jobs (if any) associated with the node.

Note that this method will only be called if either is_pure_service_node or is_mixed_service_node return True for the node.

Args
`pipeline_state`	A `PipelineState` object for an active pipeline.
`node_id`	Id of the node to stop services.