Why is feature scaling critical for SVMs?
easyAnswer
- SVMs measure distances between points (via kernels or margins).
- Features on very different scales dominate the distance computation — a feature in kilometres will completely swamp a feature in millimetres.
- Both the linear-kernel margin and the RBF kernel depend on this.
- Standardize features to mean 0 / variance 1 before fitting.
- Also matters heavily when tuning gamma — an unscaled feature makes the 'right' gamma completely different.
Check yourself — multiple choice
- SVMs are scale-invariant like trees
- Kernels/margins measure distances → features on different scales dominate; standardize first
- Scaling only matters for classification
- Scaling only affects training speed
Distance-based ⇒ features must be on comparable scales. Standardize before SVM.
#svm#preprocessing
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Which models need feature scaling and which don't?
- Why do SVMs use kernels?
- Why must you standardize features before applying L1 or L2 regularization?
- Your scikit-learn logistic regression warns 'lbfgs failed to converge'. What do you do?
- Hard-margin vs soft-margin SVM — what's the difference?
- What is a support vector, and why does the SVM only depend on them?