Custom driver for ExampleGen.
Inherits From: DRIVER_CLASS
, BaseDriver
tfx.components.FileBasedExampleGen.DRIVER_CLASS(
metadata_handler: tfx.orchestration.metadata.Metadata
)
This driver supports file based ExampleGen, e.g., for CsvExampleGen and
ImportExampleGen.
Args |
mlmd_connection
|
ML metadata connection.
|
Methods
pre_execution
View source
pre_execution(
input_dict: Dict[Text, tfx.types.Channel
],
output_dict: Dict[Text, tfx.types.Channel
],
exec_properties: Dict[Text, Any],
driver_args: tfx.orchestration.data_types.DriverArgs
,
pipeline_info: tfx.orchestration.data_types.PipelineInfo
,
component_info: tfx.orchestration.data_types.ComponentInfo
) -> tfx.orchestration.data_types.ExecutionDecision
Handle pre-execution logic.
There are four steps:
- Fetches input artifacts from metadata and checks whether uri exists.
- Registers execution.
- Decides whether a new execution is needed.
4a. If (3), prepare output artifacts.
4b. If not (3), fetch cached output artifacts.
Args |
input_dict
|
key -> Channel for inputs.
|
output_dict
|
key -> Channel for outputs. Uris of the outputs are not
assigned.
|
exec_properties
|
Dict of other execution properties.
|
driver_args
|
An instance of data_types.DriverArgs class.
|
pipeline_info
|
An instance of data_types.PipelineInfo, holding pipeline
related properties including pipeline_name, pipeline_root and run_id
|
component_info
|
An instance of data_types.ComponentInfo, holding component
related properties including component_type and component_id.
|
Returns |
data_types.ExecutionDecision object.
|
Raises |
RuntimeError
|
if any input as an empty uri.
|
resolve_exec_properties
View source
resolve_exec_properties(
exec_properties: Dict[Text, Any],
pipeline_info: tfx.orchestration.data_types.PipelineInfo
,
component_info: tfx.orchestration.data_types.ComponentInfo
) -> Dict[Text, Any]
Overrides BaseDriver.resolve_exec_properties().
View source
resolve_input_artifacts(
input_dict: Dict[Text, tfx.types.Channel
],
exec_properties: Dict[Text, Any],
driver_args: tfx.orchestration.data_types.DriverArgs
,
pipeline_info: tfx.orchestration.data_types.PipelineInfo
) -> Dict[Text, List[types.Artifact]]
Resolve input artifacts from metadata.
Subclasses might override this function for customized artifact properties
resolution logic. However please note that this function is supposed to be
called in normal cases (except head of the pipeline) since it handles
artifact info passing from upstream components.
Args |
input_dict
|
key -> Channel mapping for inputs generated in logical
pipeline.
|
exec_properties
|
Dict of other execution properties, e.g., configs.
|
driver_args
|
An instance of data_types.DriverArgs with driver
configuration properties.
|
pipeline_info
|
An instance of data_types.PipelineInfo, holding pipeline
related properties including component_type and component_id.
|
Returns |
Final artifacts that will be used in execution.
|
Raises |
ValueError
|
if in interactive mode, the given input channels have not been
resolved.
|
run
View source
run(
execution_info: tfx.orchestration.portable.data_types.ExecutionInfo
) -> driver_output_pb2.DriverOutput
Invokes the driver with inputs provided by the Launcher.
Args |
execution_info
|
a data_types.ExecutionInfo instance representing the
execution info needed for the driver execution.
|
Returns |
An DriverOutput instance.
|
View source
verify_input_artifacts(
artifacts_dict: Dict[Text, List[types.Artifact]]
) -> None
Verify that all artifacts have existing uri.
Args |
artifacts_dict
|
key -> types.Artifact for inputs.
|
Raises |
RuntimeError
|
if any input as an empty or non-existing uri.
|