The output of running an evaluation is a
which can be visualized in a Jupyter notebook by calling
To view metrics, use the
API passing the
tfma.EvalResult that was output from the evaluation run. The
metrics view is composed of three parts:
By default, all computed metrics are displayed and the columns are sorted alphabetically. Metrics selector allows the user to add / remove / reorder metrics. Simply check / uncheck metrics from the dropdown (hold Ctrl for multi-select) or type / re-arrange them directly in the input box.
Metric visualization aims to provide intuition about slices in the feature chosen. A quick filtering is available to filter out slices with small weighted sample count.
Two type of visualizations are supported:
In this view, value for the chosen metric is rendered for each slice and the slices can be sorted by the slice name or the value of another metric.
When the number of slices are small, this is the default view.
In this view, slices are broken down into buckets based on their metric values. The value(s) displayed in each bucket can be the number of slices in the bucket or the total weighted sample count for all slices in the bucket or both.
The number of buckets can be changed and logarithmic scale can be applied in the settings menu by clicking on the gear icon.
It is also possible to filter out outliers in the histogram view. Simply drag the desired range in the histogram as shown in the screenshot below.
When the number of slices are large, this is the default view.
The metric table summarizes results for all metrics chosen in metrics selector. It can be sorted by clicking on the metric name. Only slices not filtered out will be rendered.
Each plot has it own visualization that is unique to the plot. For more
information, see the relevant API documentation for the plot class. Note that
in TFMA, plots and metrics are both defined under
By convention the classes related to plots end in
Plot. To view plots, use the
API passing the
tfma.EvalResult that was output from the evaluation run.
Time Series Graphs
Time series graphs make it easy to spot trends of a specific metric over data
spans or model runs. To create a time series graph, perform multiple evaluations
(saving the output to different directories), and then load them into a
tfma.EvalResults object by calling
The results can then be displayed using
To display the graph for a specific metric, simply click on it from the dropdown list. To dismiss a graph, click on the X on the upper right corner.
Hover over any data point in the graph shows a tooltip indicating model run, data span, and metric value.