Save the date! Google I/O returns May 18-20 Register now

model_card_toolkit.ModelCardToolkit

ModelCardToolkit provides utilities to generate a ModelCard.

ModelCardToolkit is a tool for ML practitioners to create Model Cards, documentation for model information such as owners, use cases, training and evaluation data, performance, etc. A Model Card document can be displayed in output formats including HTML, Markdown, etc.

The ModelCardToolkit includes an API designed for a human-in-the-loop process to elaborate the ModelCard. If model training is integrated with ML Metadata (e.g., TFX pipelines), the ModelCardToolkit can further populate ModelCard fields by extract metadata and lineage from the model's MLMD instance.

The ModelCardToolkit organizes the ModelCard assets (e.g., structured data, plots, and UI templates) in a user-specified directory, and updates them incrementally via its API.

Example usage:

import model_card_toolkit

# Initialize the Model Card Toolkit with a path to store generate assets
model_card_output_path = ...
mct = model_card_toolkit.ModelCardToolkit(model_card_output_path)

# Initialize the ModelCard, which can be freely populated
model_card = mct.scaffold_assets()
model_card.model_details.name = 'My Model'

# Write the model card data to a JSON file
mct.update_model_card_json(model_card)

# Return the model card document as an HTML page
html = mct.export_format()

output_dir The MCT assets path where the json file and templates are written to. If not given, a temp directory is used.
mlmd_store A ml-metadata MetadataStore to retrieve metadata and lineage information about the model stored at model_uri. If given, a set of model card properties can be auto-populated from the mlmd_store.
model_uri The path to the trained model to generate model cards.

ValueError If mlmd_store is given and the model_uri cannot be resolved as a model artifact in the metadata store.

Methods

export_format

View source

Generates a model card based on the MCT assets.

Args
template_path The file path of the UI template. If not provided, the default UI template will be used.
output_file The file name of the generated model card. If not provided, the default 'model_card.html' will be used. If the file already exists, then it will be overwritten.

Returns
The model card UI.

save_mlmd

View source

Saves the model card of the model artifact with model_uri to MLMD.

scaffold_assets

View source

Generates the model cards tookit assets.

Model cards assets include the model card json file and customizable model card UI templates.

An assets directory is created if one does not already exist.

If the MCT is initialized with a mlmd_store, it further auto-populates the model cards properties as well as generating related plots such as model performance and data distributions.

Returns
A ModelCard representing the given model.

Raises
FileNotFoundError if it failed to copy the UI template files.

update_model_card_json

View source

Validates the model card and updates the JSON file in MCT assets.

If model_card.schema_version is not provided, it will assign the latest schema version to the model_card, and validate it.

Args
model_card The updated model card that users want to write back.

Raises
Error when the given model_card is invalid w.r.t. the schema.