Why must you standardize features before applying L1 or L2 regularization?
easyAnswer
- The penalty is applied uniformly to all coefficients, but the natural scale of a coefficient depends on the scale of its feature (weight * feature = signal).
- Without standardization, a feature measured in millimetres would get a huge coefficient and be over-penalized versus one in metres.
- Standardizing (mean 0, std 1) puts coefficients on comparable scales, so alpha has a consistent effect and coefficient magnitudes become interpretable.
Check yourself — multiple choice
- Standardization is only for trees
- L1/L2 penalize all coefficients equally, so features must be on the same scale for fair regularization
- It has no effect on Lasso
- Standardization removes the need for regularization
Uniform penalty ⇒ features must have comparable scale, or you over-penalize large-scale features.
#regularization#preprocessing
Practise Supervised Learning
215 interview questions in this topic.