Is adding more features always safe with gradient boosting?
mediumAnswer
- No. Boosting tolerates irrelevant features better than linear models, but there are real costs.
- Each extra feature is another thing that can break, drift or be unavailable at serving time, so the maintenance burden grows.
- Many weak, correlated features dilute the split search and can slow convergence.
- Features derived from the target or from post-event data introduce leakage that a validation split may not catch.
- And a wide model is harder to explain to a stakeholder.
- In practice, prune to the features that earn their keep by permutation importance and by whether you trust their pipeline.
Check yourself — multiple choice
- Yes, boosting handles anything
- No — extra features add pipeline fragility, drift surface, leakage risk and dilute the split search, even if accuracy barely moves
- Only linear models care
- More features always improve results
Robustness to noise features is not the same as being free: fragility and leakage risk grow.
#feature-selection#gradient-boosting
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why does L1 (Lasso) produce sparse coefficients but L2 (Ridge) does not?
- When would you use Group Lasso instead of standard Lasso?
- How is feature importance computed from a decision tree or random forest?
- In one sentence, what is Gradient Boosting?
- What is the role of the learning rate (shrinkage) in GBM?
- What did XGBoost bring on top of vanilla GBM?