mlmd.metadata_store.MetadataStore

A store for the artifact metadata.

config metadata_store_pb2.ConnectionConfig or metadata_store_pb2.MetadataStoreClientConfig. Configuration to connect to the database or the metadata store server.
enable_upgrade_migration if set to True, the library upgrades the db schema and migrates all data if it connects to an old version backend. It is ignored when using GRPC client connection config.

Methods

create_artifact_with_type

View source

Creates an artifact with a type.

This first gets the type (or creates it if it does not exist), and then puts the artifact into the database with that type.

The type_id should not be specified in the artifact (it is ignored).

Note that this is not a transaction!

  1. First, the type is created as a transaction.
  2. Then the artifact is created as a transaction.

Args
artifact the artifact to create (no id or type_id)
artifact_type the type of the new artifact (no id)

Returns
the artifact ID of the resulting type.

Raises
InvalidArgument if the type is not the same as one with the same name already in the database.

get_artifact_by_type_and_name

View source

Get the artifact of the given type and name.

The API fails if more than one artifact is found.

Args
type_name The artifact type name to look for.
artifact_name The artifact name to look for.

Returns
The Artifact matching the type and name. None if no matched Artifact was found.

get_artifact_type

View source

Gets an artifact type by name.

Args
type_name the type with that name.

Returns
The type with name type_name.

Raises

tensorflow.errors.NotFoundError: if no type exists tensorflow.errors.InternalError: if query execution fails

get_artifact_types

View source

Gets all artifact types.

Returns
A list of all known ArtifactTypes.

Raises

tensorflow.errors.InternalError: if query execution fails

get_artifact_types_by_id

View source

Gets artifact types by ID.

Args
type_ids a sequence of artifact type IDs.

Returns
A list of artifact types.

Raises
InternalError if query execution fails.

get_artifacts

View source

Gets all artifacts.

Returns
A list of all artifacts.

Raises
InternalError if query execution fails.

get_artifacts_by_context

View source

Gets all direct artifacts that a context attributes to.

Args
context_id The id of the querying context

Returns
Artifacts attributing to the context.

get_artifacts_by_id

View source

Gets all artifacts with matching ids.

The result is not index-aligned: if an id is not found, it is not returned.

Args
artifact_ids A list of artifact ids to retrieve.

Returns
Artifacts with matching ids.

get_artifacts_by_type

View source

Gets all the artifacts of a given type.

get_artifacts_by_uri

View source

Gets all the artifacts of a given uri.

get_context_by_type_and_name

View source

Get the context of the given type and context name.

The API fails if more than one contexts are found.

Args
type_name The context type name to look for.
context_name The context name to look for.

Returns
The Context matching the type and context name. None if no matched Context found.

get_context_type

View source

Gets a context type by name.

Args
type_name the type with that name.

Returns
The type with name type_name.

Raises

tensorflow.errors.NotFoundError: if no type exists tensorflow.errors.InternalError: if query execution fails

get_context_types

View source

Gets all context types.

Returns
A list of all known ContextTypes.

Raises

tensorflow.errors.InternalError: if query execution fails

get_context_types_by_id

View source

Gets context types by ID.

Args
type_ids a sequence of context type IDs.

Returns
A list of context types.

Args
type_ids ids to look for.

Raises
InternalError if query execution fails.

get_contexts

View source

Gets all contexts.

Returns
A list of all contexts.

Raises
InternalError if query execution fails.

get_contexts_by_artifact

View source

Gets all context that an artifact is attributed to.

Args
artifact_id The id of the querying artifact

Returns
Contexts that the artifact is attributed to.

get_contexts_by_execution

View source

Gets all context that an execution is associated with.

Args
execution_id The id of the querying execution

Returns
Contexts that the execution is associated with.

get_contexts_by_id

View source

Gets all contexts with matching ids.

The result is not index-aligned: if an id is not found, it is not returned.

Args
context_ids A list of context ids to retrieve.

Returns
Contexts with matching ids.

get_contexts_by_type

View source

Gets all the contexts of a given type.

Args
type_name The context type name to look for.

Returns
Contexts that matches the context type name.

get_events_by_artifact_ids

View source

Gets all events with matching artifact ids.

Args
artifact_ids a list of artifact ids.

Returns
Events with the execution IDs given.

Raises
InternalError if query execution fails.

get_events_by_execution_ids

View source

Gets all events with matching execution ids.

Args
execution_ids a list of execution ids.

Returns
Events with the execution IDs given.

Raises
InternalError if query execution fails.

get_execution_by_type_and_name

View source

Get the execution of the given type and name.

The API fails if more than one execution is found.

Args
type_name The execution type name to look for.
execution_name The execution name to look for.

Returns
The Execution matching the type and name. None if no matched Execution found.

get_execution_type

View source

Gets an execution type by name.

Args
type_name the type with that name.

Returns
The type with name type_name.

Raises

tensorflow.errors.NotFoundError: if no type exists tensorflow.errors.InternalError: if query execution fails

get_execution_types

View source

Gets all execution types.

Returns
A list of all known ExecutionTypes.

Raises

tensorflow.errors.InternalError: if query execution fails

get_execution_types_by_id

View source

Gets execution types by ID.

Args
type_ids a sequence of execution type IDs.

Returns
A list of execution types.

Args
type_ids ids to look for.

Raises
InternalError if query execution fails.

get_executions

View source

Gets all executions.

Returns
A list of all executions.

Raises
InternalError if query execution fails.

get_executions_by_context

View source

Gets all direct executions that a context associates with.

Args
context_id The id of the querying context

Returns
Executions associating with the context.

get_executions_by_id

View source

Gets all executions with matching ids.

The result is not index-aligned: if an id is not found, it is not returned.

Args
execution_ids A list of execution ids to retrieve.

Returns
Executions with matching ids.

get_executions_by_type

View source

Gets all the executions of a given type.

put_artifact_type

View source

Inserts or updates an artifact type.

If no type exists in the database with the given name, it creates a new type and returns the type_id.

If the request type with the same name already exists (let's call it stored_type), the method enforces the stored_type can be updated only when the request type is backward compatible for the already stored instances.

Backwards compatibility is violated iff:

a) there is a property where the request type and stored_type have different value type (e.g., int vs. string) b) can_add_fields = false and the request type has a new property that is not stored. c) can_omit_fields = false and stored_type has an existing property that is not provided in the request type.

Args
artifact_type the request type to be inserted or updated.
can_add_fields when true, new properties can be added; when false, returns ALREADY_EXISTS if the request type has properties that are not in stored_type.
can_omit_fields when true, stored properties can be omitted in the request type; when false, returns ALREADY_EXISTS if the stored_type has properties not in the request type.

Returns
the type_id of the response.

Raises
AlreadyExistsError If the type is not backward compatible.
InvalidArgumentError If the request type has no name, or any property value type is unknown.

put_artifacts

View source

Inserts or updates artifacts in the database.

If an artifact_id is specified for an artifact, it is an update. If an artifact_id is unspecified, it will insert a new artifact. For new artifacts, type must be specified. For old artifacts, type must be unchanged or unspecified. When the name of an artifact is given, it should be unique among artifacts of the same ArtifactType.

Args
artifacts A list of artifacts to insert or update.

Returns
A list of artifact ids index-aligned with the input.

Raises
AlreadyExistsError If artifact's name is specified and it is already used by stored artifacts of that ArtifactType.

put_attributions_and_associations

View source

Inserts attribution and association relationships in the database.

The context_id, artifact_id, and execution_id must already exist. If the relationship exists, this call does nothing. Once added, the relationships cannot be modified.

Args
attributions A list of attributions to insert.
associations A list of associations to insert.

put_context_type

View source

Inserts or updates a context type.

If no type exists in the database with the given name, it creates a new type and returns the type_id.

If the request type with the same name already exists (let's call it stored_type), the method enforces the stored_type can be updated only when the request type is backward compatible for the already stored instances.

Backwards compatibility is violated iff:

a) there is a property where the request type and stored_type have different value type (e.g., int vs. string) b) can_add_fields = false and the request type has a new property that is not stored. c) can_omit_fields = false and stored_type has an existing property that is not provided in the request type.

Args
context_type the request type to be inserted or updated.
can_add_fields when true, new properties can be added; when false, returns ALREADY_EXISTS if the request type has properties that are not in stored_type.
can_omit_fields when true, stored properties can be omitted in the request type; when false, returns ALREADY_EXISTS if the stored_type has properties not in the request type.

Returns
the type_id of the response.

Raises
AlreadyExistsError If the type is not backward compatible.
InvalidArgumentError If the request type has no name, or any property value type is unknown.

put_contexts

View source

Inserts or updates contexts in the database.

If an context_id is specified for an context, it is an update. If an context_id is unspecified, it will insert a new context. For new contexts, type must be specified. For old contexts, type must be unchanged or unspecified. The name of a context cannot be empty, and it should be unique among contexts of the same ContextType.

Args
contexts A list of contexts to insert or update.

Returns
A list of context ids index-aligned with the input.

Raises
InvalidArgumentError If name of the new contexts are empty.
AlreadyExistsError If name of the new contexts already used by stored contexts of that ContextType.

put_events

View source

Inserts events in the database.

The execution_id and artifact_id must already exist. Once created, events cannot be modified.

Args
events A list of events to insert.

put_execution

View source

Inserts or updates an Execution with artifacts, events and contexts.

If an execution_id, artifact_id or context_id is specified, it is an update, otherwise it does an insertion.

Args
execution The execution to be created or updated.
artifact_and_events a pair of Artifact and Event that the execution uses or generates. The event's execution id or artifact id can be empty, as the artifact or execution may not be stored beforehand. If given, the ids must match with the paired Artifact and the input execution.
contexts The Contexts that the execution should be assotiated with and the artifacts should be attributed to.

Returns
the execution id, the list of artifact's id, and the list of context's id.

put_execution_type

View source

Inserts or updates an execution type.

If no type exists in the database with the given name, it creates a new type and returns the type_id.

If the request type with the same name already exists (let's call it stored_type), the method enforces the stored_type can be updated only when the request type is backward compatible for the already stored instances.

Backwards compatibility is violated iff:

a) there is a property where the request type and stored_type have different value type (e.g., int vs. string) b) can_add_fields = false and the request type has a new property that is not stored. c) can_omit_fields = false and stored_type has an existing property that is not provided in the request type.

Args
execution_type the request type to be inserted or updated.
can_add_fields when true, new properties can be added; when false, returns ALREADY_EXISTS if the request type has properties that are not in stored_type.
can_omit_fields when true, stored properties can be omitted in the request type; when false, returns ALREADY_EXISTS if the stored_type has properties not in the request type.

Returns
the type_id of the response.

Raises
AlreadyExistsError If the type is not backward compatible.
InvalidArgumentError If the request type has no name, or any property value type is unknown.

put_executions

View source

Inserts or updates executions in the database.

If an execution_id is specified for an execution, it is an update. If an execution_id is unspecified, it will insert a new execution. For new executions, type must be specified. For old executions, type must be unchanged or unspecified. When the name of an execution is given, it should be unique among executions of the same ExecutionType.

Args
executions A list of executions to insert or update.

Returns
A list of execution ids index-aligned with the input.

Raises
AlreadyExistsError If execution's name is specified and it is already used by stored executions of that ExecutionType.