Which models need feature scaling and which don't?
easyAnswer
- Scale features for models that use distances or gradient descent: k-NN, k-means, SVM, PCA, linear/logistic regression with regularization, and neural networks.
- Tree-based models (decision trees, random forest, gradient boosting) don't need scaling because splits are threshold-based and invariant to monotonic transforms.
Check yourself — multiple choice
- Trees require standardization
- k-NN, SVM, and neural nets benefit from scaling; trees do not
- All models need MinMax scaling
- Only regression models need scaling
Distance- and gradient-based models need scaling; trees are scale-invariant.
#preprocessing#features
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- StandardScaler vs MinMaxScaler vs RobustScaler — how do you choose?
- What is target leakage and how do you prevent it?
- What is the curse of dimensionality?
- What is VIF and when do you worry about multicollinearity?
- How do you handle strong multicollinearity in a linear model?
- What's the main risk of polynomial regression?