tfx.components.example_gen.csv_example_gen.executor.Executor

View source on GitHub

Generic TFX CSV example gen executor.

Inherits From: BaseExampleGenExecutor

Child Classes

class Context

Methods

Do

View source

Take input data source and generates TF Example splits.

Args
input_dict Input dict from input key to a list of Artifacts. Depends on detailed example gen implementation.
output_dict Output dict from output key to a list of Artifacts.

  • examples: splits of tf examples.
exec_properties A dict of execution properties. Depends on detailed example gen implementation.
  • input: JSON string of example_gen_pb2.Input instance, providing input configuration.
  • output: JSON string of example_gen_pb2.Output instance, providing output configuration.
  • Returns
    None

    GenerateExamplesByBeam

    View source

    Converts input source to TF example splits based on configs.

    Custom ExampleGen executor should provide GetInputSourceToExamplePTransform for converting input split to TF Examples. Overriding this 'GenerateExamplesByBeam' method instead if complex logic is need, e.g., custom spliting logic.

    Args
    pipeline beam pipeline.
    input_dict Input dict from input key to a list of Artifacts. Depends on detailed example gen implementation.
    exec_properties A dict of execution properties. Depends on detailed example gen implementation.

    • input: JSON string of example_gen_pb2.Input instance, providing input configuration.
    • output: JSON string of example_gen_pb2.Output instance, providing output configuration.

    Returns
    Dict of beam PCollection with split name as key, each PCollection is a single output split that contains serialized TF Examples.

    GetInputSourceToExamplePTransform

    View source

    Returns PTransform for CSV to TF examples.