tfx.orchestration.beam.beam_dag_runner.BeamDagRunner

View source on GitHub

Class BeamDagRunner

Tfx runner on Beam.

Inherits From: TfxRunner

__init__

View source

__init__(
    beam_orchestrator_args=None,
    config=None
)

Initializes BeamDagRunner as a TFX orchestrator.

Args:

  • beam_orchestrator_args: beam args for the beam orchestrator. Note that this is different from the beam_pipeline_args within additional_pipeline_args, which is for beam pipelines in components.
  • config: Optional pipeline config for customizing the launching of each component.

Methods

run

View source

run(tfx_pipeline)

Deploys given logical pipeline on Beam.

Args:

  • tfx_pipeline: Logical pipeline containing pipeline args and components.