tfx.orchestration.pipeline.Pipeline

View source on GitHub

Logical TFX pipeline object.

tfx.orchestration.pipeline.Pipeline(
    pipeline_name: Text,
    pipeline_root: Text,
    metadata_connection_config: Optional[metadata_store_pb2.ConnectionConfig] = None,
    components: Optional[List[base_component.BaseComponent]] = None,
    enable_cache: Optional[bool] = False,
    beam_pipeline_args: Optional[List[Text]] = None,
    **kwargs
)

pipeline_name name of the pipeline;
pipeline_root path to root directory of the pipeline;
metadata_connection_config the config to connect to ML metadata.
components a list of components in the pipeline (optional only for backward compatible purpose to be used with deprecated PipelineDecorator).
enable_cache whether or not cache is enabled for this run.
beam_pipeline_args Beam pipeline args for beam jobs within executor. Executor will use beam DirectRunner as Default.
**kwargs additional kwargs forwarded as pipeline args.

pipeline_args kwargs used to create real pipeline implementation. This is forwarded to PipelineRunners instead of consumed in this class. This should include:

  • pipeline_name: Required. The unique name of this pipeline.
  • pipeline_root: Required. The root of the pipeline outputs.
components logical components of this pipeline.
pipeline_info An instance of data_types.PipelineInfo that contains basic properties of the pipeline.
enable_cache whether or not cache is enabled for this run.
metadata_connection_config the config to connect to ML metadata.
beam_pipeline_args Beam pipeline args for beam jobs within executor. Executor will use beam DirectRunner as Default.
additional_pipeline_args other pipeline args.