EasyDeepLearn
Supervised Learning · section 7 of 18

SVMs & kernels

5 interview questions on svms & kernels, each answered in full. Free to read, no account needed.

Why do SVMs use kernels?

medium
  • Kernels let SVMs learn nonlinear boundaries without explicitly computing high-dimensional features — the kernel trick evaluates inner products in an implicit feature space.
  • Common kernels: linear (baseline), polynomial, RBF (default nonlinear choice), sigmoid.
  • RBF works well when the boundary is smooth but nonlinear; tune C and gamma.
#svm#kernelsPermalink & quiz →

Hard-margin vs soft-margin SVM — what's the difference?

medium
  • Hard-margin SVM finds the hyperplane that separates classes with the largest margin — only works when classes are linearly separable, otherwise there is no feasible solution.
  • Soft-margin adds slack variables ξi\xi_{i} ≥ 0 that let some points violate the margin (xi in the margin, xi > 1 misclassified) and penalizes them via C * sum xi.
  • C is a regularization knob: large C → few violations, high variance; small C → many violations, high bias.
  • Almost every real SVM is soft-margin.
#svmPermalink & quiz →

What does gamma control in an RBF-kernel SVM?

medium
  • The RBF kernel is exp⁡(−γ  ⋅  ∣∣x  −  x∣∣2)\operatorname{exp}( - \gamma\; \cdot \; \mid \mid x\; - \;x \mid \mid 2). gamma is 1/(2⋅σ2)1 / (2 \cdot \sigma^{2}) — the inverse of the kernel bandwidth.
  • Small gamma → wide bell, each point influences a large region → smooth boundary, high bias.
  • Large gamma → narrow bell, each point influences only near neighbours → wiggly boundary, high variance, overfits easily.
  • Tune gamma jointly with C on a log-log grid.
  • 'auto' = 1/nfeatures1 / n_{\mathrm{features}}, 'scale' = 1/(nfeatures  ⋅  Xvar())1 / (n_{\mathrm{features}}\; \cdot \;X\mathrm{var}()) in scikit-learn.
#svm#kernels#hyperparameter-tuningPermalink & quiz →

How do SVMs handle multi-class problems?

medium
  • Native SVM is binary.
  • Multi-class strategies: (1) One-vs-Rest (OvR): K binary SVMs, one per class vs the rest.
  • Simple but scores aren't comparable across classes; ties can occur.
  • (2) One-vs-One (OvO): K(K-1)/2 pairwise SVMs; each test point voted on by all binary classifiers.
  • Better for SVMs because each pair sees a balanced problem, and it scales better than OvR when class sizes are uneven. scikit-learn's SVC uses OvO by default.
#svm#multiclassPermalink & quiz →

Why is feature scaling critical for SVMs?

easy
  • SVMs measure distances between points (via kernels or margins).
  • Features on very different scales dominate the distance computation — a feature in kilometres will completely swamp a feature in millimetres.
  • Both the linear-kernel margin and the RBF kernel depend on this.
  • Standardize features to mean 0 / variance 1 before fitting.
  • Also matters heavily when tuning gamma — an unscaled feature makes the 'right' gamma completely different.
#svm#preprocessingPermalink & quiz →

Practise Supervised Learning

All 18 sectionsFlashcardsQuiz