|View source on GitHub|
FlipeRate computes the rate at which predicitons between models switch.
tfma.metrics.BooleanFlipRates( threshold: float = _DEFAULT_FLIP_RATE_THRESHOLD, flip_rate_name: str = FLIP_RATE_NAME, neg_to_neg_flip_rate_name: str = NEG_TO_NEG_FLIP_RATE_NAME, neg_to_pos_flip_rate_name: str = NEG_TO_POS_FLIP_RATE_NAME, pos_to_neg_flip_rate_name: str = POS_TO_NEG_FLIP_RATE_NAME, pos_to_pos_flip_rate_name: str = POS_TO_POS_FLIP_RATE_NAME )
Given a pair of models and a threshold for converting continuous model outputs into boolean predictions, this metric will produce three numbers (keyed by separate MetricKeys):
- (symmetric) flip rate: The number of times the boolean predictions don't match, regardless of the direction of the flip.
- negative-to-positive flip rate: The rate at which the baseline model's boolean prediction is negative but the candidate model's is positive.
- positive-to-negative flip rate: The rate at which the baseline model's boolean prediction is positive but the candidate model's is negative.
computations( eval_config: Optional[
tfma.EvalConfig] = None, schema: Optional[schema_pb2.Schema] = None, model_names: Optional[List[str]] = None, output_names: Optional[List[str]] = None, sub_keys: Optional[List[Optional[SubKey]]] = None, aggregation_type: Optional[AggregationType] = None, class_weights: Optional[Dict[int, float]] = None, example_weighted: bool = False, query_key: Optional[str] = None ) ->
Creates computations associated with metric.
get_config() -> Dict[str, Any]
Returns serializable config.