model_card_toolkit.QuantitativeAnalysis

The quantitative analysis of a model.

Inherits From: BaseModelCardField

model_card_toolkit.QuantitativeAnalysis(
    performance_metrics: List[model_card_toolkit.PerformanceMetric] = dataclasses.field(default_factory=list),
    graphics: model_card_toolkit.GraphicsCollection = dataclasses.field(default_factory=GraphicsCollection),
    _proto_type: dataclasses.InitVar[type(model_card_pb2.QuantitativeAnalysis)] = model_card_pb2.QuantitativeAnalysis
)

Identify relevant performance metrics and display values. Let’s say you’re interested in displaying the accuracy and false positive rate (FPR) of a cat vs. dog classification model. Assuming you have already computed both metrics, both overall and per-class, you can specify metrics like so:

model_card.quantitative_analysis.performance_metrics = [
  {'type': 'accuracy', 'value': computed_accuracy},
  {'type': 'accuracy', 'value': cat_accuracy, 'slice': 'cat'},
  {'type': 'accuracy', 'value': dog_accuracy, 'slice': 'dog'},
  {'type': 'fpr', 'value': computed_fpr},
  {'type': 'fpr', 'value': cat_fpr, 'slice': 'cat'},
  {'type': 'fpr', 'value': dog_fpr, 'slice': 'dog'},
]

performance_metrics The performance metrics being reported.
graphics A collection of visualizations of model performance.

Methods

clear

View source

clear()

Clear the subfields of this BaseModelCardField.

copy_from_proto

View source

copy_from_proto(
    proto: message.Message
) -> 'BaseModelCardField'

Copies the contents of the model card proto into current object.

merge_from_proto

View source

merge_from_proto(
    proto: message.Message
) -> 'BaseModelCardField'

Merges the contents of the model card proto into current object.

to_dict

View source

to_dict() -> Dict[str, Any]

Convert your model card to a python dictionary.

to_json

View source

to_json() -> str

Convert this class object to json.

to_proto

View source

to_proto() -> message.Message

Convert this class object to the proto.

__eq__

__eq__(
    other
)

__len__

View source

__len__() -> int

Returns the number of items in a field. Ignores None values recursively, so the length of a field that only contains another field that has all None values would be 0.