Multi-class cross-entropy vs one-vs-rest — pros and cons?
mediumAnswer
- Cross-entropy on softmax: mutually exclusive classes, single loss, joint probability calibration → standard for classification.
- One-vs-rest with K sigmoids + BCE: allows multi-label, imbalanced-per-label handling, per-class thresholding at inference.
- Cross-entropy is more parameter-efficient for many classes; OVR is essential for multi-label but gives up joint calibration.
Check yourself — multiple choice
- Same objective
- Softmax CE: exclusive classes, joint calibration; OVR sigmoids: multi-label, per-class thresholds
- OVR is always better
- Softmax handles multi-label
Softmax CE for exclusive multi-class; K-sigmoids BCE for multi-label.
#losses
Practise Deep Learning
214 interview questions in this topic.
Related questions
- Why compute softmax + cross-entropy jointly via log-sum-exp?
- How should the output head be designed for a regression task with a strictly positive target?
- How is the output head different for multi-label vs multi-class classification?
- What is label smoothing and why does it help?
- How does YOLO / SSD / RetinaNet differ from two-stage detection?
- Briefly, what do BLEU and ROUGE measure?