How do you handle strong multicollinearity in a linear model?
mediumAnswer
- (1) Drop one of the correlated features (highest VIF first).
- (2) Combine them via PCA or domain-driven aggregation.
- (3) Switch to Ridge regression — the L2 penalty stabilizes coefficients under collinearity by shrinking correlated features together.
- (4) Collect more data if the collinearity is sample-specific.
- Note: multicollinearity does not bias predictions, only inference on individual coefficients, so if you only need predictions you can leave it alone.
Check yourself — multiple choice
- Multicollinearity biases predictions and must always be removed
- Drop features, use PCA, or switch to ridge regression
- Only Lasso solves multicollinearity perfectly
- You must always drop half of the features
Drop, combine, or shrink (Ridge). Multicollinearity hurts inference, not prediction.
#linear-regression#regularization#features
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- What's the main risk of polynomial regression?
- What is VIF and when do you worry about multicollinearity?
- When should you add interaction features to a linear model?
- When would you use Group Lasso instead of standard Lasso?
- How do you detect and fix overfitting?
- L1 vs L2 regularization — what's the difference?