One-vs-Rest, One-vs-One, and softmax — how do you choose for multi-class?
mediumAnswer
- One-vs-Rest trains K binary classifiers (class k vs the rest); simple and calibrates per-class, but scores aren't jointly normalized.
- One-vs-One trains K(K-1)/2 binary models between every pair; more models but each sees a balanced binary problem, useful for SVMs. Softmax / multinomial logistic regression jointly models all classes with a single objective; the natural choice for logistic regression and neural nets and it produces coherent probabilities that sum to one.
Check yourself — multiple choice
- One-vs-One requires only 2 models
- Softmax models all classes jointly and produces coherent probabilities
- One-vs-Rest is only for SVMs
- Softmax cannot be used with logistic regression
Softmax: joint model, coherent probabilities. OvR: K binary. OvO: pairwise (often for SVMs).
#logistic-regression#classification#multiclass
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- 'multinomial' vs 'ovr' setting in scikit-learn's LogisticRegression — what changes?
- Why do we use cross-entropy (log loss) instead of MSE for classification?
- Write out the softmax function and its main property.
- When would LDA outperform logistic regression?
- Macro vs micro vs weighted averaging in multi-class metrics — how do you choose?
- What is top-k accuracy and when do you use it?