tfx.v1.orchestration.experimental.KubeflowV2DagRunner

Kubeflow V2 pipeline runner (currently for managed pipelines).

Used in the notebooks

Used in the tutorials

Builds a pipeline job spec in json format based on TFX pipeline DSL object.

config An KubeflowV2DagRunnerConfig object to specify runtime configuration when running the pipeline in Kubeflow.
output_dir An optional output directory into which to output the pipeline definition files. Defaults to the current working directory.
output_filename An optional output file name for the pipeline definition file. The file output format will be a JSON-serialized or YAML-serialized PipelineJob pb message. Defaults to 'pipeline.json'.

Methods

run

Compiles a pipeline DSL object into pipeline file.

Args
pipeline TFX pipeline object.
parameter_values mapping from runtime parameter names to its values.
write_out set to True to actually write out the file to the place designated by output_dir and output_filename. Otherwise return the JSON-serialized pipeline job spec.

Returns
Returns the JSON/YAML pipeline job spec.

Raises
RuntimeError if trying to write out to a place occupied by an existing file.

set_exit_handler

Set exit handler components for the Kubeflow V2(Vertex AI) dag runner.

This feature is currently experimental without backward compatibility gaurantee.

Args
exit_handler exit handler component.