How do you evaluate a multi-label classifier?
mediumAnswer
- Common metrics: (a) Hamming loss = fraction of misclassified labels (per-label error) — lenient; (b) exact match / subset accuracy = fraction of examples where ALL labels are correctly predicted — strict; (c) per-label F1 with macro/micro averaging — the workhorse; (d) label ranking metrics (mean average precision, coverage error) when scores matter more than binary decisions.
- Always report per-label metrics too, especially when label frequencies differ.
Check yourself — multiple choice
- Multi-label metrics are the same as multi-class
- Hamming loss, subset accuracy, per-label F1, label-ranking metrics
- Only accuracy works
- You must reduce it to binary
Hamming (per-label), subset accuracy (all labels), per-label F1 macro/micro, ranking metrics.
#metrics#classification#multilabel
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Precision vs recall — when do you optimize each?
- How do you read a confusion matrix and derive the key metrics?
- Why is accuracy a bad primary metric for many real-world problems?
- What is F-beta score and when is F1 not enough?
- Macro vs micro vs weighted averaging in multi-class metrics — how do you choose?
- What is Cohen's kappa and why is it useful?