Module: tfma.metrics

Init module for TensorFlow Model Analysis metrics.

Classes

class AUC: Approximates the AUC (Area under the curve) of the ROC or PR curves.

class AUCCurve: An enumeration.

class AUCPrecisionRecall: Alias for AUC(curve='PR').

class AUCSummationMethod: An enumeration.

class AttributionsMetric: Base type for attribution metrics.

class BalancedAccuracy: Balanced accuracy (BA).

class BinaryAccuracy: Calculates how often predictions match binary labels.

class BinaryCrossEntropy: Calculates the binary cross entropy.

class BooleanFlipRates: FlipRate is the rate at which predictions between models switch.

class COCOAveragePrecision: Confusion matrix at thresholds.

class COCOAverageRecall: Average recall metric for object detection.

class COCOMeanAveragePrecision: Mean average precision for object detections.

class COCOMeanAverageRecall: Mean Average recall metric for object detection.

class Calibration: Calibration.

class CalibrationPlot: Calibration plot.

class CategoricalCrossEntropy: Calculates the categorical cross entropy.

class CoefficientOfDiscrimination: Coefficient of discrimination metric.

class ConfusionMatrixAtThresholds: Confusion matrix at thresholds.

class ConfusionMatrixPlot: Confusion matrix plot.

class DerivedMetricComputation: DerivedMetricComputation derives its result from other computations.

class DiagnosticOddsRatio: Diagnostic odds ratio (DOR).

class ExactMatch: Exact Match Metric.

class ExampleCount: Example count.

class F1Score: F1 score.

class FN: Alias for FalseNegatives.

class FNR: Alias for MissRate.

class FP: Alias for FalsePositives.

class FPR: Alias for FallOut.

class FallOut: Fall-out (FPR).

class FalseDiscoveryRate: False discovery rate (FDR).

class FalseNegatives: Calculates the number of false negatives.

class FalseOmissionRate: False omission rate (FOR).

class FalsePositives: Calculates the number of false positives.

class FowlkesMallowsIndex: Fowlkes-Mallows index (FM).

class Informedness: Informedness or bookmaker informedness (BM).

class Markedness: Markedness (MK) or deltaP.

class MatthewsCorrelationCoefficient: Matthews corrrelation coefficient (MCC).

class MaxRecall: Computes the max recall of the predictions with respect to the labels.

class Mean: Mean metric.

class MeanAbsoluteAttributions: Mean aboslute attributions metric.

class MeanAbsoluteError: Calculates the mean of absolute error between labels and predictions.

class MeanAbsolutePercentageError: Calculates the mean of absolute percentage error.

class MeanAttributions: Mean attributions metric.

class MeanLabel: Mean label.

class MeanPrediction: Mean prediction.

class MeanSquaredError: Calculates the mean of squared error between labels and predictions.

class MeanSquaredLogarithmicError: Calculates the mean of squared logarithmic error.

class Metric: Metric wraps a set of metric computations.

class MetricComputation: MetricComputation represents one or more metric computations.

class MetricKey: A MetricKey uniquely identifies a metric.

class MinLabelPosition: Min label position metric.

class MissRate: Miss rate (FNR).

class MultiClassConfusionMatrixAtThresholds: Multi-class confusion matrix metrics at thresholds.

class MultiClassConfusionMatrixPlot: Multi-class confusion matrix plot.

class MultiLabelConfusionMatrixPlot: Multi-label confusion matrix.

class NDCG: NDCG (normalized discounted cumulative gain) metric.

class NPV: Alias for NegativePredictiveValue.

class NegativeLikelihoodRatio: Negative likelihood ratio (LR-).

class NegativePredictiveValue: Negative predictive value (NPV).

class ObjectDetectionConfusionMatrixPlot: Object Detection Confusion matrix plot.

class ObjectDetectionMaxRecall: Computes the max recall of the predictions with respect to the labels.

class ObjectDetectionPrecision: Computes the precision of the predictions with respect to the labels.

class ObjectDetectionPrecisionAtRecall: Computes best precision where recall is >= specified value.

class ObjectDetectionRecall: Computes the recall of the predictions with respect to the labels.

class ObjectDetectionThresholdAtRecall: Computes maximum threshold where recall is >= specified value.

class PPV: Alias for Precision.

class PlotKey: A PlotKey is a metric key that uniquely identifies a plot.

class PositiveLikelihoodRatio: Positive likelihood ratio (LR+).

class Precision: Computes the precision of the predictions with respect to the labels.

class PrecisionAtRecall: Computes best precision where recall is >= specified value.

class Preprocessor: Preprocessor wrapper for preprocessing data in the metric computation.

class Prevalence: Prevalence.

class PrevalenceThreshold: Prevalence threshold (PT).

class QueryStatistics: Query statistic metrics.

class Recall: Computes the recall of the predictions with respect to the labels.

class RecallAtPrecision: Computes best recall where precision is >= specified value.

class RelativeCoefficientOfDiscrimination: Relative coefficient of discrimination metric.

class ScoreDistributionPlot: Score distribution plot.

class SemanticSegmentationConfusionMatrix: Computes confusion matrices for semantic segmentation.

class SemanticSegmentationFalsePositive: Calculates the true postive for semantic segmentation.

class SemanticSegmentationTruePositive: Calculates the true postive for semantic segmentation.

class SensitivityAtSpecificity: Computes best sensitivity where specificity is >= specified value.

class SetMatchPrecision: Computes precision for sets of labels and predictions.

class SetMatchRecall: Computes recall for sets of labels and predictions.

class Specificity: Specificity (TNR) or selectivity.

class SpecificityAtSensitivity: Computes best specificity where sensitivity is >= specified value.

class SquaredPearsonCorrelation: Squared pearson correlation (r^2) metric.

class StandardMetricInputs: Standard inputs used by most metric computations.

class SubKey: A SubKey identifies a sub-types of metrics and plots.

class SymmetricPredictionDifference: PredictionDifference computes the avg pointwise diff between models.

class TN: Alias for TrueNegatives.

class TNR: Alias for Specificity.

class TP: Alias for TruePositives.

class TPR: Alias for Recall.

class ThreatScore: Threat score or critical success index (TS or CSI).

class TotalAbsoluteAttributions: Total absolute attributions metric.

class TotalAttributions: Total attributions metric.

class TrueNegatives: Calculates the number of true negatives.

class TruePositives: Calculates the number of true positives.

class WeightedExampleCount: Weighted example count (deprecated - use ExampleCount).

Functions

CombinedFeaturePreprocessor(...): Returns preprocessor for incl combined features in StandardMetricInputs.

FeaturePreprocessor(...): Returns preprocessor for including features in StandardMetricInputs.

default_binary_classification_specs(...): Returns default metric specs for binary classification problems.

default_multi_class_classification_specs(...): Returns default metric specs for multi-class classification problems.

default_regression_specs(...): Returns default metric specs for for regression problems.

has_attributions_metrics(...): Returns true if any of the metrics_specs have attributions metrics.

merge_per_key_computations(...): Wraps create_computations_fn to be called separately for each key.

metric_thresholds_from_metrics_specs(...): Returns thresholds associated with given metrics specs.

specs_from_metrics(...): Returns specs for tf_keras.metrics/losses or tfma.metrics classes.

to_label_prediction_example_weight(...): Yields label, prediction, and example weights for use in calculations.

to_standard_metric_inputs(...): Verifies extract keys and converts extracts to StandardMetricInputs.

Type Aliases

MetricComputations

MetricsDict

NO_PREDICTED_CLASS_ID -1