Join the SIG TFX-Addons community and help make TFX even better!

tfx.orchestration.portable.beam_executor_operator.BeamExecutorOperator

BeamExecutorOperator handles Beam based executor's init and execution.

Inherits From: BaseExecutorOperator

executor_spec The specification of how to initialize the executor.
platform_config The specification of how to allocate resource for the executor.

extra_flags Extra flags that will pass to Beam executors. It come from two sources in the order:

  1. The extra_flags set in the executor spec.
  2. The flags passed in when starting the program by users or by other systems. The interpretation of these flags relying on the executor implementation.
beam_pipeline_args Beam specific arguments that will pass to Beam executors. It comes from beam_pipeline_args set in the Beam executor spec.

Methods

run_executor

View source

Invokes executors given input from the Launcher.

Args
execution_info A wrapper of the details of this execution.

Returns
The output from executor.

SUPPORTED_EXECUTOR_SPEC_TYPE [<class 'tfx.proto.orchestration.executable_spec_pb2.BeamExecutableSpec'>]
SUPPORTED_PLATFORM_CONFIG_TYPE []