Attend the Women in ML Symposium on December 7 Register now


Stay organized with collections Save and categorize content based on your preferences.

Compute the Expected Calibration Error (ECE).

This method implements equation (3) in [1]. In this equation the probability of the decided label being correct is used to estimate the calibration property of the predictor.


[1]: Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger, On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning (ICML 2017). arXiv:1706.04599

num_bins int, number of probability bins, e.g. 10.
logits Tensor, (n,nlabels), with logits for n instances and nlabels.
labels_true Tensor, (n,), with tf.int32 or tf.int64 elements containing ground truth class labels in the range [0,nlabels].
labels_predicted Tensor, (n,), with tf.int32 or tf.int64 elements containing decisions of the predictive system. If None, we will use the argmax decision using the logits.
name Python str name prefixed to Ops created by this function.

ece Tensor, scalar, tf.float32.