tfx.orchestration.pipeline.Pipeline

View source on GitHub

Logical TFX pipeline object.

pipeline_name name of the pipeline;
pipeline_root path to root directory of the pipeline;
metadata_connection_config the config to connect to ML metadata.
components a list of components in the pipeline (optional only for backward compatible purpose to be used with deprecated PipelineDecorator).
enable_cache whether or not cache is enabled for this run.
beam_pipeline_args Beam pipeline args for beam jobs within executor. Executor will use beam DirectRunner as Default.
**kwargs additional kwargs forwarded as pipeline args.

pipeline_args kwargs used to create real pipeline implementation. This is forwarded to PipelineRunners instead of consumed in this class. This should include:

  • pipeline_name: Required. The unique name of this pipeline.
  • pipeline_root: Required. The root of the pipeline outputs.
components logical components of this pipeline.
pipeline_info An instance of data_types.PipelineInfo that contains basic properties of the pipeline.
enable_cache whether or not cache is enabled for this run.
metadata_connection_config the config to connect to ML metadata.
beam_pipeline_args Beam pipeline args for beam jobs within executor. Executor will use beam DirectRunner as Default.
additional_pipeline_args other pipeline args.