tfx.orchestration.pipeline.Pipeline

Logical TFX pipeline object.

tfx.orchestration.pipeline.Pipeline(
    pipeline_name: Text,
    pipeline_root: Text,
    metadata_connection_config: Optional[metadata.ConnectionConfigType] = None,
    components: Optional[List[base_node.BaseNode]] = None,
    enable_cache: Optional[bool] = False,
    beam_pipeline_args: Optional[List[Text]] = None,
    platform_config: Optional[message.Message] = None,
    execution_mode: Optional[tfx.orchestration.pipeline.ExecutionMode] = tfx.orchestration.pipeline.ExecutionMode.SYNC,
    **kwargs
)

pipeline_name Name of the pipeline;
pipeline_root Path to root directory of the pipeline;
metadata_connection_config The config to connect to ML metadata.
components A list of components in the pipeline (optional only for backward compatible purpose to be used with deprecated PipelineDecorator).
enable_cache Whether or not cache is enabled for this run.
beam_pipeline_args Pipeline arguments for Beam powered Components.
platform_config Pipeline level platform config, in proto form.
execution_mode The execution mode of the pipeline, can be SYNC or ASYNC.
**kwargs Additional kwargs forwarded as pipeline args.

pipeline_args Kwargs used to create real pipeline implementation. This is forwarded to PipelineRunners instead of consumed in this class. This should include:

  • pipeline_name: Required. The unique name of this pipeline.
  • pipeline_root: Required. The root of the pipeline outputs.
components Logical components of this pipeline. When read, this is in topologically sorted order.
pipeline_info An instance of data_types.PipelineInfo that contains basic properties of the pipeline.
enable_cache Whether or not cache is enabled for this run.
metadata_connection_config The config to connect to ML metadata.
execution_mode Execution mode of the pipeline. Currently only support synchronous execution mode.
beam_pipeline_args Pipeline arguments for Beam powered Components.
platform_config Pipeline level platform config, in proto form.
additional_pipeline_args Other pipeline args.