tfx.components.common_nodes.importer_node.ImporterDriver

Driver for Importer.

Inherits From: DRIVER_CLASS

Methods

pre_execution

View source

Handle pre-execution logic.

There are four steps:

  1. Fetches input artifacts from metadata and checks whether uri exists.
  2. Registers execution.
  3. Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args
input_dict key -> Channel for inputs.
output_dict key -> Channel for outputs. Uris of the outputs are not assigned.
exec_properties Dict of other execution properties.
driver_args An instance of data_types.DriverArgs class.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
data_types.ExecutionDecision object.

Raises
RuntimeError if any input as an empty uri.

resolve_exec_properties

View source

Resolve execution properties.

Subclasses might override this function for customized execution properties resolution logic.

Args
exec_properties Original execution properties passed in.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
Final execution properties that will be used in execution.

resolve_input_artifacts

View source

Resolve input artifacts from metadata.

Subclasses might override this function for customized artifact properties resolution logic. However please note that this function is supposed to be called in normal cases (except head of the pipeline) since it handles artifact info passing from upstream components.

Args
input_dict key -> Channel mapping for inputs generated in logical pipeline.
exec_properties Dict of other execution properties, e.g., configs.
driver_args An instance of data_types.DriverArgs with driver configuration properties.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including component_type and component_id.

Returns
Final artifacts that will be used in execution.

Raises
ValueError if in interactive mode, the given input channels have not been resolved.

verify_input_artifacts

View source

Verify that all artifacts have existing uri.

Args
artifacts_dict key -> types.Artifact for inputs.

Raises
RuntimeError if any input as an empty or non-existing uri.