Utilities for ExampleGen components.
Functions
calculate_splits_fingerprint_span_and_version(...)
: Calculates the fingerprint of files in a URI matching split patterns.
date_to_span_number(...)
: Calculated span from date as number of days since unix epoch.
dict_to_example(...)
: Converts dict to tf example.
generate_output_split_names(...)
: Return output split name based on input and output config.
make_default_input_config(...)
: Returns default input config.
make_default_output_config(...)
: Returns default output config based on input config.
span_number_to_date(...)
: Given a span number, convert it to the corresponding calendar date.
Other Members |
DATE_SPECS
|
|
DAY_SPEC
|
'{DD}'
|
EXAMPLES_KEY
|
'examples'
|
FINGERPRINT_PROPERTY_NAME
|
'input_fingerprint'
|
INPUT_BASE_KEY
|
'input_base'
|
INPUT_CONFIG_KEY
|
'input_config'
|
MONTH_SPEC
|
'{MM}'
|
OUTPUT_CONFIG_KEY
|
'output_config'
|
OUTPUT_DATA_FORMAT_KEY
|
'output_data_format'
|
PAYLOAD_FORMAT_PROPERTY_NAME
|
'payload_format'
|
RANGE_CONFIG_KEY
|
'range_config'
|
SPAN_FULL_REGEX
|
'{SPAN}|{SPAN:(?P<width>.*?)}'
|
SPAN_PROPERTY_NAME
|
'span'
|
SPAN_SPEC
|
'{SPAN}'
|
SPAN_SPEC_WIDTH_REGEX
|
'{SPAN:(?P<width>.*?)}'
|
UNIX_EPOCH_DATE
|
|
VERSION_FULL_REGEX
|
'{VERSION}|{VERSION:(?P<width>.*?)}'
|
VERSION_PROPERTY_NAME
|
'version'
|
VERSION_SPEC
|
'{VERSION}'
|
VERSION_SPEC_WIDTH_REGEX
|
'{VERSION:(?P<width>.*?)}'
|
YEAR_SPEC
|
'{YYYY}'
|