Join the SIG TFX-Addons community and help make TFX even better!

tfx.components.example_gen.input_processor.FileBasedInputProcessor

Custom InputProcessor for file based ExampleGen driver.

Inherits From: InputProcessor

input_base_uri The base path from which files will be searched.
splits An iterable collection of example_gen_pb2.Input.Split objects.
range_config An instance of range_config_pb2.RangeConfig, defines the rules for span resolving.

Methods

get_input_fingerprint

View source

Returns the fingerprint for a certain Version of a certain Span.

get_latest_span

View source

Resolves the latest Span information.

get_latest_version

View source

Resolves the latest Version of a Span.

get_pattern_for_span_version

View source

Return pattern with Span and Version spec filled.

resolve_span_and_version

View source

Resolves Span and Version information.

If a pattern has the {SPAN} placeholder or the Date spec placeholders, {YYYY}, {MM}, and {DD}, and optionally, the {VERSION} placeholder, attempts to find aligned values that results in all splits having the target span and most recent version for that span.

Returns
A Tuple of [target_span, target_version], where:

  1. target_span is either the value matched with the {SPAN} placeholder, the value mapped from matching the calendar date with the date placeholders {YYYY}, {MM}, {DD} or 0 if a placeholder wasn't specified.
  2. target_version is either the value matched with the {VERSION} placeholder, or None if the placeholder wasn't specified.