How do SVMs handle multi-class problems?
mediumAnswer
- Native SVM is binary.
- Multi-class strategies: (1) One-vs-Rest (OvR): K binary SVMs, one per class vs the rest.
- Simple but scores aren't comparable across classes; ties can occur.
- (2) One-vs-One (OvO): K(K-1)/2 pairwise SVMs; each test point voted on by all binary classifiers.
- Better for SVMs because each pair sees a balanced problem, and it scales better than OvR when class sizes are uneven. scikit-learn's SVC uses OvO by default.
Check yourself — multiple choice
- SVMs support multi-class natively via softmax
- One-vs-Rest or One-vs-One (default in scikit-learn's SVC)
- Only via error-correcting codes
- They can't be extended beyond binary classification
OvR or OvO wrappers around K binary SVMs. sklearn.svm.SVC uses OvO.
#svm#multiclass
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why do SVMs use kernels?
- One-vs-Rest, One-vs-One, and softmax — how do you choose for multi-class?
- 'multinomial' vs 'ovr' setting in scikit-learn's LogisticRegression — what changes?
- Hard-margin vs soft-margin SVM — what's the difference?
- What is a support vector, and why does the SVM only depend on them?
- How does the C parameter in an SVM affect the fit?