Why does L1 (Lasso) produce sparse coefficients but L2 (Ridge) does not?
mediumAnswer
- Geometrically, the L1 constraint region is a diamond with corners on the axes, so the loss contour tends to intersect the constraint at a corner where some coefficients are exactly zero.
- The L2 region is a sphere, so intersections are typically off-axis — coefficients become small but not zero.
- Algebraically, L1's subgradient at zero is a range containing zero, so a solver can 'stick' coefficients at exactly zero.
Check yourself — multiple choice
- Both L1 and L2 produce sparse solutions
- L1's diamond-shaped constraint has corners on the axes, which drives coefficients to exactly zero
- L2 is more aggressive than L1
- Sparsity comes only from cross-validation
L1's non-smooth corners at the axes cause exact zero coefficients; L2's smooth sphere does not.
#regularization#linear-models#feature-selection
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- L1 vs L2 regularization — what's the difference?
- Why does Ridge regression give more stable coefficients than OLS?
- When is Elastic Net better than pure Lasso or Ridge?
- How do you choose the regularization strength (lambda / alpha)?
- In one sentence, what does the LARS algorithm compute?
- When would you use Group Lasso instead of standard Lasso?