tfx.components.FileBasedExampleGen.DRIVER_CLASS

Custom driver for ExampleGen.

Inherits From: DRIVER_CLASS, BaseDriver

This driver supports file based ExampleGen, e.g., for CsvExampleGen and ImportExampleGen.

mlmd_connection ML metadata connection.
pipeline_info The information of the pipeline that this driver is in.
pipeline_node The specification of the node that this driver is in.

Methods

pre_execution

View source

Handle pre-execution logic.

There are four steps:

  1. Fetches input artifacts from metadata and checks whether uri exists.
  2. Registers execution.
  3. Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args
input_dict key -> Channel for inputs.
output_dict key -> Channel for outputs. Uris of the outputs are not assigned.
exec_properties Dict of other execution properties.
driver_args An instance of data_types.DriverArgs class.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
data_types.ExecutionDecision object.

Raises
RuntimeError if any input as an empty uri.

resolve_exec_properties

View source

Overrides BaseDriver.resolve_exec_properties().

resolve_input_artifacts

View source

Resolve input artifacts from metadata.

Subclasses might override this function for customized artifact properties resolution logic. However please note that this function is supposed to be called in normal cases (except head of the pipeline) since it handles artifact info passing from upstream components.

Args
input_dict key -> Channel mapping for inputs generated in logical pipeline.
exec_properties Dict of other execution properties, e.g., configs.
driver_args An instance of data_types.DriverArgs with driver configuration properties.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including component_type and component_id.

Returns
Final artifacts that will be used in execution.

Raises
ValueError if in interactive mode, the given input channels have not been resolved.

run

View source

Invokes the driver with inputs provided by the Launcher.

Args
input_dict The defult input_dict resolved by the launcher.
output_dict The default output_dict resolved by the launcher.
exec_properties The default exec_properties resolved by the launcher.

Returns
An DriverOutput instance.

verify_input_artifacts

View source

Verify that all artifacts have existing uri.

Args
artifacts_dict key -> types.Artifact for inputs.

Raises
RuntimeError if any input as an empty or non-existing uri.