tfx.components.FileBasedExampleGen.DRIVER_CLASS

View source on GitHub

Custom driver for ExampleGen.

Inherits From: DRIVER_CLASS

This driver supports file based ExampleGen, it registers external file path as an artifact, e.g., for CsvExampleGen and ImportExampleGen.

Methods

pre_execution

View source

Handle pre-execution logic.

There are four steps:

  1. Fetches input artifacts from metadata and checks whether uri exists.
  2. Registers execution.
  3. Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args
input_dict key -> Channel for inputs.
output_dict key -> Channel for outputs. Uris of the outputs are not assigned.
exec_properties Dict of other execution properties.
driver_args An instance of data_types.DriverArgs class.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
data_types.ExecutionDecision object.

Raises
RuntimeError if any input as an empty uri.

resolve_exec_properties

View source

Resolve execution properties.

Subclasses might override this function for customized execution properties resolution logic.

Args
exec_properties Original execution properties passed in.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
Final execution properties that will be used in execution.

resolve_input_artifacts

View source

Overrides BaseDriver.resolve_input_artifacts().

verify_input_artifacts

View source

Verify that all artifacts have existing uri.

Args
artifacts_dict key -> types.Artifact for inputs.

Raises
RuntimeError if any input as an empty or non-existing uri.