TensorFlow 2.0 RC is available

# tf.contrib.metrics.cohen_kappa

Calculates Cohen's kappa.

``````tf.contrib.metrics.cohen_kappa(
labels,
predictions_idx,
num_classes,
weights=None,
metrics_collections=None,
name=None
)
``````

Cohen's kappa is a statistic that measures inter-annotator agreement.

The `cohen_kappa` function calculates the confusion matrix, and creates three local variables to compute the Cohen's kappa: `po`, `pe_row`, and `pe_col`, which refer to the diagonal part, rows and columns totals of the confusion matrix, respectively. This value is ultimately returned as `kappa`, an idempotent operation that is calculated by

``````pe = (pe_row * pe_col) / N
k = (sum(po) - sum(pe)) / (N - sum(pe))
``````

For estimation of the metric over a stream of data, the function creates an `update_op` operation that updates these variables and returns the `kappa`. `update_op` weights each prediction by the corresponding value in `weights`.

Class labels are expected to start at 0. E.g., if `num_classes` was three, then the possible labels would be [0, 1, 2].

If `weights` is `None`, weights default to 1. Use weights of 0 to mask values.

NOTE: Equivalent to `sklearn.metrics.cohen_kappa_score`, but the method doesn't support weighted matrix yet.

#### Args:

• `labels`: 1-D `Tensor` of real labels for the classification task. Must be one of the following types: int16, int32, int64.
• `predictions_idx`: 1-D `Tensor` of predicted class indices for a given classification. Must have the same type as `labels`.
• `num_classes`: The possible number of labels.
• `weights`: Optional `Tensor` whose shape matches `predictions`.
• `metrics_collections`: An optional list of collections that `kappa` should be added to.
• `updates_collections`: An optional list of collections that `update_op` should be added to.
• `name`: An optional variable_scope name.

#### Returns:

• `kappa`: Scalar float `Tensor` representing the current Cohen's kappa.
• `update_op`: `Operation` that increments `po`, `pe_row` and `pe_col` variables appropriately and whose value matches `kappa`.

#### Raises:

• `ValueError`: If `num_classes` is less than 2, or `predictions` and `labels` have mismatched shapes, or if `weights` is not `None` and its shape doesn't match `predictions`, or if either `metrics_collections` or `updates_collections` are not a list or tuple.
• `RuntimeError`: If eager execution is enabled.