tfx.components.FileBasedExampleGen.DRIVER_CLASS

View source on GitHub

Class DRIVER_CLASS

Custom driver for ExampleGen.

Inherits From: DRIVER_CLASS

Aliases: tfx.components.CsvExampleGen.DRIVER_CLASS, tfx.components.ImportExampleGen.DRIVER_CLASS, tfx.components.example_gen.component.FileBasedExampleGen.DRIVER_CLASS, tfx.components.example_gen.csv_example_gen.component.CsvExampleGen.DRIVER_CLASS, tfx.components.example_gen.driver.Driver, tfx.components.example_gen.import_example_gen.component.ImportExampleGen.DRIVER_CLASS

This driver supports file based ExampleGen, it registers external file path as an artifact, e.g., for CsvExampleGen and ImportExampleGen.

__init__

View source

__init__(metadata_handler)

Initialize self. See help(type(self)) for accurate signature.

Methods

pre_execution

View source

pre_execution(
    input_dict,
    output_dict,
    exec_properties,
    driver_args,
    pipeline_info,
    component_info
)

Handle pre-execution logic.

There are four steps:

  1. Fetches input artifacts from metadata and checks whether uri exists.
  2. Registers execution.
  3. Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args:

  • input_dict: key -> Channel for inputs.
  • output_dict: key -> Channel for outputs. Uris of the outputs are not assigned.
  • exec_properties: Dict of other execution properties.
  • driver_args: An instance of data_types.DriverArgs class.
  • pipeline_info: An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
  • component_info: An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns:

data_types.ExecutionDecision object.

Raises:

  • RuntimeError: if any input as an empty uri.

resolve_exec_properties

View source

resolve_exec_properties(
    exec_properties,
    pipeline_info,
    component_info
)

Resolve execution properties.

Subclasses might override this function for customized execution properties resolution logic.

Args:

  • exec_properties: Original execution properties passed in.
  • pipeline_info: An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
  • component_info: An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns:

Final execution properties that will be used in execution.

resolve_input_artifacts

View source

resolve_input_artifacts(
    input_channels,
    exec_properties,
    driver_args,
    pipeline_info
)

Overrides BaseDriver.resolve_input_artifacts().

verify_input_artifacts

View source

verify_input_artifacts(artifacts_dict)

Verify that all artifacts have existing uri.

Args:

  • artifacts_dict: key -> types.Artifact for inputs.

Raises:

  • RuntimeError: if any input as an empty or non-existing uri.