What is Cohen's kappa and why is it useful?

medium

Answer

$\kappa\; = \;(p_{o}\; - \;p_{e})\; / \;(1\; - \;p_{e})$
$It corrects accuracy for the baseline of random guessing given the class prior.$
$Range: [-1, 1]. 0 = no better than chance; 1 = perfect.$
$Rules of thumb: 0.2 slight, 0.4 fair, 0.6 substantial, 0.8+ almost perfect.$
$Useful for imbalanced classification and inter-annotator agreement.$

Check yourself — multiple choice

Kappa is the same as accuracy
Accuracy corrected for chance given class marginals; useful on imbalanced problems
Kappa only works for regression
Kappa is always between 0 and 100

$\kappa\; = \;(\mathrm{accuracy}\;\;\mathrm{chance})\; / \;(1\;\;\mathrm{chance})$ — chance-corrected agreement.

#metrics#classification#imbalance

215 interview questions in this topic.