Join the SIG TFX-Addons community and help make TFX even better!

tfx.orchestration.beam.beam_dag_runner.BeamDagRunner

Tfx runner on Beam.

Inherits From: TfxRunner

tfx.orchestration.beam.beam_dag_runner.BeamDagRunner(
    beam_orchestrator_args: Optional[List[Text]] = None,
    config: Optional[tfx.orchestration.config.pipeline_config.PipelineConfig] = None
)

beam_orchestrator_args Deprecated beam args for the beam orchestrator. Note that this is different from the beam_pipeline_args within additional_pipeline_args, which is for beam pipelines in components. If this option is used, the legacy non-IR-based BeamDagRunner will be constructed.
config Deprecated optional pipeline config for customizing the launching of each component. Defaults to pipeline config that supports InProcessComponentLauncher and DockerComponentLauncher. If this option is used, the legacy non-IR-based BeamDagRunner will be constructed.

Methods

run

View source

run(
    pipeline: Union[pipeline_pb2.Pipeline, tfx.orchestration.pipeline.Pipeline]
) -> None

Deploys given logical pipeline on Beam.

Args
pipeline Logical pipeline in IR format.