Random forest vs gradient boosting — which do you pick and why?
mediumAnswer
- Random forest averages many deep trees trained on bootstrapped samples with random feature subsets — it reduces variance and is robust with little tuning.
- Gradient boosting (XGBoost, LightGBM, CatBoost) fits trees sequentially, each correcting residuals — it usually gives better accuracy on tabular data but needs more tuning and is more prone to overfitting.
- For a strong baseline on tabular data: GBDT wins.
- For a fast, safe baseline: random forest.
Check yourself — multiple choice
- Random forest boosts sequentially
- Gradient boosting averages independent trees
- GBDT usually beats RF on tabular data but needs tuning
- Both are always identical
GBDT is sequential and typically stronger; RF is parallel and safer out-of-the-box.
#trees#ensembles
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Gini impurity, entropy, and classification error — which do trees actually use?
- How do decision trees handle categorical features?
- Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?
- How does cost-complexity (CCP) pruning work?
- How do decision trees handle missing values?
- What criterion does a regression tree use to choose splits?