tfx.components.FileBasedExampleGen.DRIVER_CLASS

Custom Driver for file based ExampleGen, e.g., ImportExampleGen.

Inherits From: Driver, DRIVER_CLASS, BaseDriver

View aliases

Main aliases

tfx.components.CsvExampleGen.DRIVER_CLASS, tfx.components.ImportExampleGen.DRIVER_CLASS, tfx.components.example_gen.component.FileBasedExampleGen.DRIVER_CLASS, tfx.components.example_gen.csv_example_gen.component.CsvExampleGen.DRIVER_CLASS, tfx.components.example_gen.driver.FileBasedDriver, tfx.components.example_gen.import_example_gen.component.ImportExampleGen.DRIVER_CLASS

tfx.components.FileBasedExampleGen.DRIVER_CLASS(
    metadata_handler: tfx.orchestration.metadata.Metadata
)

Args
`mlmd_connection`	ML metadata connection.

Methods

`get_input_processor`

View source

get_input_processor(
    splits: Iterable[example_gen_pb2.Input.Split],
    range_config: Optional[range_config_pb2.RangeConfig] = None,
    input_base_uri: Optional[Text] = None
) -> tfx.components.example_gen.input_processor.InputProcessor

Returns FileBasedInputProcessor.

`pre_execution`

View source

pre_execution(
    input_dict: Dict[Text, tfx.types.Channel],
    output_dict: Dict[Text, tfx.types.Channel],
    exec_properties: Dict[Text, Any],
    driver_args: tfx.orchestration.data_types.DriverArgs,
    pipeline_info: tfx.orchestration.data_types.PipelineInfo,
    component_info: tfx.orchestration.data_types.ComponentInfo
) -> tfx.orchestration.data_types.ExecutionDecision

Handle pre-execution logic.

There are four steps:

Fetches input artifacts from metadata and checks whether uri exists.
Registers execution.
Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args
`input_dict`	key -> Channel for inputs.
`output_dict`	key -> Channel for outputs. Uris of the outputs are not assigned.
`exec_properties`	Dict of other execution properties.
`driver_args`	An instance of data_types.DriverArgs class.
`pipeline_info`	An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
`component_info`	An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
data_types.ExecutionDecision object.

Raises
`RuntimeError`	if any input as an empty uri.

`resolve_exec_properties`

View source

resolve_exec_properties(
    exec_properties: Dict[Text, Any],
    pipeline_info: tfx.orchestration.data_types.PipelineInfo,
    component_info: tfx.orchestration.data_types.ComponentInfo
) -> Dict[Text, Any]

Overrides BaseDriver.resolve_exec_properties().

`resolve_input_artifacts`

View source

resolve_input_artifacts(
    input_dict: Dict[Text, tfx.types.Channel],
    exec_properties: Dict[Text, Any],
    driver_args: tfx.orchestration.data_types.DriverArgs,
    pipeline_info: tfx.orchestration.data_types.PipelineInfo
) -> Dict[Text, List[types.Artifact]]

Resolve input artifacts from metadata.

Subclasses might override this function for customized artifact properties resolution logic. However please note that this function is supposed to be called in normal cases (except head of the pipeline) since it handles artifact info passing from upstream components.

Args
`input_dict`	key -> Channel mapping for inputs generated in logical pipeline.
`exec_properties`	Dict of other execution properties, e.g., configs.
`driver_args`	An instance of data_types.DriverArgs with driver configuration properties.
`pipeline_info`	An instance of data_types.PipelineInfo, holding pipeline related properties including component_type and component_id.

Returns
Final artifacts that will be used in execution.

Raises
`ValueError`	if in interactive mode, the given input channels have not been resolved.

`run`

View source

run(
    execution_info: tfx.orchestration.portable.data_types.ExecutionInfo
) -> driver_output_pb2.DriverOutput

Invokes the driver with inputs provided by the Launcher.

Args
`execution_info`	a `data_types.ExecutionInfo` instance representing the execution info needed for the driver execution.

Returns
An DriverOutput instance.

`verify_input_artifacts`

View source

verify_input_artifacts(
    artifacts_dict: Dict[Text, List[types.Artifact]]
) -> None

Verify that all artifacts have existing uri.

Args
`artifacts_dict`	key -> types.Artifact for inputs.

Raises
`RuntimeError`	if any input as an empty or non-existing uri.