|View source on GitHub|
Helper class to handle metadata I/O.
Initialize self. See help(type(self)) for accurate signature.
Returns underlying MetadataStore.
RuntimeError: if this instance is not in enter state.
__exit__( exc_type, exc_value, exc_tb )
check_artifact_state( artifact, expected_states )
fetch_previous_result_artifacts( output_dict, execution_id )
Fetches output with artifact ids produced by a previous run.
output_dict: a dict from name to a list of output Artifact objects.
execution_id: the id of the execution that produced the outputs.
Original output_dict with artifact id inserted.
RuntimeError: path change without clean metadata.
Get all runs for a given pipeline name.
pipeline_name: name of the pipeline.
A List of run id.
Get components execution states for a given pipeline.
pipeline_info: target pipeline's information.
A Dict of component id to its state mapping.
previous_execution( input_artifacts, exec_properties, pipeline_info, component_info )
Gets eligible previous execution that takes the same inputs.
An eligible execution should take the same inputs, execution properties and with the same pipeline and component properties.
input_artifacts: inputs used by the run.
exec_properties: execution properties used by the run.
pipeline_info: info of the current pipeline run.
component_info: info of the current component.
Execution id of previous run that takes the input dict. None if not found.
Publish a list of artifacts if any is not already published.
publish_execution( execution_id, input_dict, output_dict, state=EXECUTION_STATE_COMPLETE )
Publish an execution with input and output artifacts info.
execution_id: id of execution to be published.
input_dict: inputs artifacts used by the execution with id ready.
output_dict: output artifacts produced by the execution without id.
state: optional state of the execution, default to be EXECUTION_STATE_COMPLETE.
Updated outputs with artifact ids.
RuntimeError: If any of the following happens:
- Execution state not valid for publish
- Input artifact id missing
- Output artifact id missing when execution state is EXECUTION_STATE_CASHED
register_execution( exec_properties, pipeline_info, component_info, run_context_id=None )
Create a new execution in metadata.
exec_properties: the execution properties of the execution.
pipeline_info: optional pipeline info of the execution.
component_info: optional component info of the execution.
run_context_id: context id for current run, link it with execution if provided.
execution id of the new execution.
Create or get the context for current pipeline run.
pipeline_info: pipeline information for current run.
context id of the current run.
search_artifacts( artifact_name, pipeline_name, run_id, producer_component_id )
Search artifacts that matches given info.
artifact_name: the name of the artifact that set by producer component. The name is logged both in artifacts and the events when the execution being published.
pipeline_name: the name of the pipeline that produces the artifact
run_id: the run id of the pipeline run that produces the artifact
producer_component_id: the id of the component that produces the artifact
A list of Artifacts that matches the given info
RuntimeError: when no matching execution is found given producer info.
update_artifact_state( artifact, new_state )