tfx.components.base.base_component.BaseComponent.DRIVER_CLASS

View source on GitHub

Class DRIVER_CLASS

BaseDriver is the base class of all custom drivers.

Aliases:

This can also be used as the default driver of a component if no custom logic is needed.

Attributes:

  • _metadata_handler: An instance of Metadata.

__init__

View source

__init__(metadata_handler)

Initialize self. See help(type(self)) for accurate signature.

Methods

pre_execution

View source

pre_execution(
    input_dict,
    output_dict,
    exec_properties,
    driver_args,
    pipeline_info,
    component_info
)

Handle pre-execution logic.

There are four steps:

  1. Fetches input artifacts from metadata and checks whether uri exists.
  2. Registers execution.
  3. Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args:

  • input_dict: key -> Channel for inputs.
  • output_dict: key -> Channel for outputs. Uris of the outputs are not assigned.
  • exec_properties: Dict of other execution properties.
  • driver_args: An instance of data_types.DriverArgs class.
  • pipeline_info: An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
  • component_info: An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns:

data_types.ExecutionDecision object.

Raises:

  • RuntimeError: if any input as an empty uri.

resolve_exec_properties

View source

resolve_exec_properties(
    exec_properties,
    component_info
)

Resolve execution properties.

Subclasses might override this function for customized execution properties resolution logic.

Args:

  • exec_properties: Original execution properties passed in.
  • component_info: An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns:

Final execution properties that will be used in execution.

resolve_input_artifacts

View source

resolve_input_artifacts(
    input_dict,
    exec_properties,
    driver_args,
    pipeline_info
)

Resolve input artifacts from metadata.

Subclasses might override this function for customized artifact properties resolution logic. However please note that this function is supposed to be called in normal cases (except head of the pipeline) since it handles artifact info passing from upstream components.

Args:

  • input_dict: key -> Channel mapping for inputs generated in logical pipeline.
  • exec_properties: Dict of other execution properties, e.g., configs.
  • driver_args: An instance of data_types.DriverArgs with driver configuration properties.
  • pipeline_info: An instance of data_types.PipelineInfo, holding pipeline related properties including component_type and component_id.

Returns:

Final execution properties that will be used in execution.

Raises:

  • RuntimeError: for Channels that do not contain any artifact. This will be reverted once we support Channel-based input resolution.
  • ValueError: if in interactive mode, the given input channels have not been resolved.