In practice, how do you handle correlated features in regression?
mediumAnswer
- (1) Domain knowledge: keep the more meaningful one (age vs birthyear).
- (2) Combine: sum, difference, or PCA / PLS.
- (3) Ridge: L2 shrinks correlated coefficients toward each other without dropping.
- (4) Lasso: L1 arbitrarily picks one of a correlated pair.
- (5) Elastic net: middle ground — spreads coefficients across correlated groups.
- (6) Feature importance-based selection.
- Rule: if inference matters, ridge / elastic net + domain pruning; if pure prediction, tree ensembles usually don't care about collinearity.
Check yourself — multiple choice
- Ignore
- Domain knowledge / PCA / PLS / ridge (L2) / elastic net; lasso picks one arbitrarily; trees usually robust to collinearity
- Always drop
- Same as heteroscedasticity
Correlated features: domain / PCA / ridge / elastic net; trees robust.
#regression#assumptions
Practise Statistics Fundamentals
215 interview questions in this topic.