tfx.components.transform.executor.Executor

View source on GitHub

Transform executor.

Inherits From: BaseExecutor

tfx.components.transform.executor.Executor(
    context: Optional[tfx.components.base.base_executor.BaseExecutor.Context] = None
)

Child Classes

class Context

Methods

Do

View source

Do(
    input_dict: Dict[Text, List[tfx.types.Artifact]],
    output_dict: Dict[Text, List[tfx.types.Artifact]],
    exec_properties: Dict[Text, Any]
) -> None

TensorFlow Transform executor entrypoint.

This implements BaseExecutor.Do() and is invoked by orchestration systems. This is not inteded for manual usage or further customization. Please use the Transform() function which takes an input format with no artifact dependency.

Args
input_dict Input dict from input key to a list of artifacts, including:
output_dict Output dict from key to a list of artifacts, including:
  • transform_output: Output of 'tf.Transform', which includes an exported Tensorflow graph suitable for both training and serving;
  • transformed_examples: Materialized transformed examples, which includes both 'train' and 'eval' splits.
    • exec_properties A dict of execution properties, including either one of:
  • module_file: The file path to a python module file, from which the 'preprocessing_fn' function will be loaded.
  • preprocessing_fn: The module path to a python function that implements 'preprocessing_fn'.

    • Returns
    None

    Transform

    View source

    Transform(
    inputs: Mapping[Text, Any],
    outputs: Mapping[Text, Any],
    status_file: Text
) -> None

    Executes on request.

    This is the implementation part of transform executor. This is intended for using or extending the executor without artifact dependency.

    Args
    inputs A dictionary of labelled input values, including:

    • labels.COMPUTE_STATISTICS_LABEL: Whether compute statistics.
    • labels.SCHEMA_PATH_LABEL: Path to schema file.
    • labels.EXAMPLES_DATA_FORMAT_LABEL: Example data format.
    • labels.ANALYZE_DATA_PATHS_LABEL: Paths or path patterns to analyze data.
    • labels.ANALYZE_PATHS_FILE_FORMATS_LABEL: File formats of paths to analyze data.
    • labels.TRANSFORM_DATA_PATHS_LABEL: Paths or path patterns to transform data.
    • labels.TRANSFORM_PATHS_FILE_FORMATS_LABEL: File formats of paths to transform data.
    • labels.MODULE_FILE: Path to a Python module that contains the preprocessing_fn, optional.
    • labels.PREPROCESSING_FN: Path to a Python function that implements preprocessing_fn, optional.
    • labels.USE_TFXIO_LABEL: Whether use the TFXIO-based TFT APIs.
    outputs A dictionary of labelled output values, including:
  • labels.PER_SET_STATS_OUTPUT_PATHS_LABEL: Paths to statistics output, optional.
  • labels.TRANSFORM_METADATA_OUTPUT_PATH_LABEL: A path to TFTransformOutput output.
  • labels.TRANSFORM_MATERIALIZE_OUTPUT_PATHS_LABEL: Paths to transform materialization.
  • labels.TEMP_OUTPUT_LABEL: A path to temporary directory.
    • status_file Where the status should be written (not yet implemented)