tfma.analyze_raw_data

Runs TensorFlow model analysis on a pandas.DataFrame.

This function allows you to use TFMA with Pandas DataFrames. The dataframe must include a 'predicted' column for the predicted label and a 'label' column for the actual label.

In addition to a DataFrame, this function requires an eval_config, a tfma.EvalConfig object containing various configuration parameters (see config.proto for a comprehensive list)...

  • the metrics to compute
  • the slices to compute metrics on
  • the DataFrame's column names for example labels and predictions ('label' and 'prediction' by default)
  • confidence interval options

This function returns a tfma.EvalResult, which contains TFMA's computed metrics and can be used to generate plots with tfma.view.render_slicing_metrics.

Example usage:

model_specs = [
  config.ModelSpec(
      prediction_key='prediction',
      label_key='label')
]
config.options.compute_confidence_intervals.value
metrics_specs = [
    config.MetricsSpec(metrics=[
      config.MetricConfig(class_name='Accuracy'),
      config.MetricConfig(class_name='ExampleCount')
    ])
]
slicing_specs = [
    config.SlicingSpec(),  # the empty slice represents the overall dataset
    config.SlicingSpec(feature_keys=['language'])
]
eval_config = config.EvalConfig(
    model_specs=model_specs,
    metrics_specs=metrics_specs,
    slicing_specs=slicing_specs)
result = model_eval_lib.analyze_raw_data(df, eval_config)
tfma.view.render_slicing_metrics(result)

# Example with Fairness Indicators
from tensorflow_model_analysis.addons.fairness.post_export_metrics import
fairness_indicators
from tensorflow_model_analysis.addons.fairness.view import widget_view
add_metrics_callbacks = [
    tfma.post_export_metrics.fairness_indicators(thresholds=[0.25, 0.5, 0.75])
]
result = model_eval_lib.analyze_raw_data(
    data=df,
    metrics_specs=metrics_specs,
    slicing_specs=slicing_specs,
    add_metric_callbacks=add_metrics_callbacks
)
widget_view.render_fairness_indicator(result)

data A pandas.DataFrame, where rows correspond to examples and columns correspond to features. One column must indicate a row's predicted label, and one column must indicate a row's actual label.
eval_config A tfma.EvalConfig, which contains various configuration parameters including metrics, slices, and label/prediction column names.
output_path Path to write EvalResult to.
add_metric_callbacks Optional list of metric callbacks (if used).

A tfma.EvalResult to extract metrics or generate visualizations from.

KeyError If the prediction or label columns are not found within the DataFrame.