Why does Random Forest sample features at each split, not just once per tree?
mediumAnswer
- Per-split feature subsampling forces different trees to explore different features and different splits.
- Without it, if one feature is very predictive, every tree would keep splitting on it near the root and become highly correlated — averaging correlated trees barely reduces variance.
- Sampling features at each split de-correlates the ensemble.
- Default for classification, d/3 for regression is a well-tested rule of thumb.
Check yourself — multiple choice
- To make training slower
- To de-correlate trees so averaging actually reduces variance
- To reduce memory only
- To bias trees toward specific features
Feature subsampling de-correlates trees, which is the point of the ensemble.
#random-forest#bagging#ensembles
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How does bagging reduce variance?
- How do Extra Trees differ from Random Forests?
- What is the out-of-bag (OOB) score in Random Forests?
- Random forest vs gradient boosting — which do you pick and why?
- Which Random Forest hyperparameters actually matter for tuning?
- Are Random Forest probability estimates well-calibrated?