How is the output head different for multi-label vs multi-class classification?
easyAnswer
- Multi-class: one softmax head, one loss (categorical cross-entropy) — labels are mutually exclusive.
- Multi-label: K independent sigmoid heads, one binary cross-entropy loss per label, summed — a document can have multiple tags simultaneously.
- Do not use softmax for multi-label: it forces the K probabilities to sum to 1, which contradicts independence.
Check yourself — multiple choice
- Use softmax for multi-label
- Multi-class: softmax + cross-entropy; multi-label: K sigmoids + summed BCE
- They're identical
- Multi-label uses regression heads
Softmax for exclusive classes; K independent sigmoids for multi-label.
#fundamentals#losses
Practise Deep Learning
214 interview questions in this topic.
Related questions
- Why compute softmax + cross-entropy jointly via log-sum-exp?
- How should the output head be designed for a regression task with a strictly positive target?
- MSE vs MAE for regression — when is each preferred?
- ReLU vs sigmoid vs GELU — when do you use each?
- In one sentence, what is backpropagation?
- State the universal approximation theorem in one sentence and its practical caveat.