EasyDeepLearn

What does gamma control in an RBF-kernel SVM?

medium

Answer

  • The RBF kernel is exp⁡(−γ  ⋅  ∣∣x  −  x∣∣2)\operatorname{exp}( - \gamma\; \cdot \; \mid \mid x\; - \;x \mid \mid 2). gamma is 1/(2⋅σ2)1 / (2 \cdot \sigma^{2}) — the inverse of the kernel bandwidth.
  • Small gamma → wide bell, each point influences a large region → smooth boundary, high bias.
  • Large gamma → narrow bell, each point influences only near neighbours → wiggly boundary, high variance, overfits easily.
  • Tune gamma jointly with C on a log-log grid.
  • 'auto' = 1/nfeatures1 / n_{\mathrm{features}}, 'scale' = 1/(nfeatures  ⋅  Xvar())1 / (n_{\mathrm{features}}\; \cdot \;X\mathrm{var}()) in scikit-learn.
Check yourself — multiple choice
  • gamma is the learning rate
  • gamma controls RBF bandwidth — small γ  =  smooth\gamma\; = \;\mathrm{smooth} boundary; large γ  =  wiggly  /  overfit\gamma\; = \;\mathrm{wiggly}\; / \;\mathrm{overfit}
  • gamma is only used for linear kernels
  • gamma is fixed and can't be tuned

γ  =  1/(2σ2)\gamma\; = \;1 / (2{\sigma}^{2}): small = smooth, large = wiggly. Tune jointly with C.

#svm#kernels#hyperparameter-tuning

Practise Supervised Learning

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions