|View source on GitHub|
Approximates the AUC (Area under the curve) of the ROC or PR curves.
tfma.metrics.AUC( num_thresholds: Optional[int] = None, curve: str = 'ROC', summation_method: str = 'interpolation', name: Optional[str] = None, thresholds: Optional[Union[float, List[float]]] = None, top_k: Optional[int] = None, class_id: Optional[int] = None )
The AUC (Area under the curve) of the ROC (Receiver operating characteristic; default) or PR (Precision Recall) curves are quality measures of binary classifiers. Unlike the accuracy, and like cross-entropy losses, ROC-AUC and PR-AUC evaluate all the operational points of a model.
This class approximates AUCs using a Riemann sum. During the metric accumulation phase, predictions are accumulated within predefined buckets by value. The AUC is then computed by interpolating per-bucket averages. These buckets define the evaluated operational points.
This metric uses
false_negatives to compute the AUC. To discretize the AUC curve, a linearly
spaced set of thresholds is used to compute pairs of recall and precision
values. The area under the ROC-curve is therefore computed using the height of
the recall values by the false positive rate, while the area under the
PR-curve is the computed using the height of the precision values by the
This value is ultimately returned as
auc, an idempotent operation that
computes the area under a discretized curve of precision versus recall values
(computed using the aforementioned variables). The
controls the degree of discretization with larger numbers of thresholds more
closely approximating the true AUC. The quality of the approximation may vary
dramatically depending on
thresholds parameter can be
used to manually specify thresholds which split the predictions more evenly.
For a best approximation of the real AUC,
predictions should be distributed
approximately uniformly in the range [0, 1]. The quality of the AUC
approximation may be poor if this is not the case. Setting
to 'minoring' or 'majoring' can help quantify the error in the approximation
by providing lower or upper bound estimate of the AUC.
None, weights default to 1.
sample_weight of 0 to mask values.
||(Optional) Defaults to 10000. The number of thresholds to use when discretizing the roc curve. Values must be > 1.|
||(Optional) Specifies the name of the curve to be computed, 'ROC' [default] or 'PR' for the Precision-Recall-curve.|
(Optional) Specifies the Riemann summation method used. 'interpolation'
(default) applies mid-point summation scheme for
||(Optional) string name of the metric instance.|
(Optional) A list of floating point values to use as the
thresholds for discretizing the curve. If set, the
||(Optional) Used with a multi-class model to specify that the top-k values should be used to compute the confusion matrix. The net effect is that the non-top-k values are set to -inf and the matrix is then constructed from the average TP, FP, TN, FN across the classes. When top_k is used, metrics_specs.binarize settings must not be present. Only one of class_id or top_k should be configured.|
||(Optional) Used with a multi-class model to specify which class to compute the confusion matrix for. When class_id is used, metrics_specs.binarize settings must not be present. Only one of class_id or top_k should be configured.|
Whether to compute confidence intervals for this metric.
Note that this may not completely remove the computational overhead involved in computing a given metric. This is only respected by the jackknife confidence interval method.
computations( eval_config: Optional[
tfma.EvalConfig] = None, schema: Optional[schema_pb2.Schema] = None, model_names: Optional[List[str]] = None, output_names: Optional[List[str]] = None, sub_keys: Optional[List[Optional[SubKey]]] = None, aggregation_type: Optional[AggregationType] = None, class_weights: Optional[Dict[int, float]] = None, example_weighted: bool = False, query_key: Optional[str] = None ) ->
Creates computations associated with metric.
get_config() -> Dict[str, Any]
Returns serializable config.