How do Extra Trees differ from Random Forests?
mediumAnswer
- Two changes: (1) each tree is trained on the whole training set (no bootstrapping); (2) splits are chosen randomly — for each candidate feature, a random split threshold is drawn rather than the optimal one, and the best of those random splits is picked.
- Result: more randomness, more bias, less variance, and *faster* training.
- Typically slightly worse than RF on well-tuned problems but sometimes better on very noisy data and always faster.
Check yourself — multiple choice
- Extra Trees add L2 regularization
- They use random split thresholds and no bootstrapping — more variance reduction, faster training
- Extra Trees are only for regression
- They are the same as Random Forests
Extra Trees = random thresholds + no bootstrapping = faster, more randomized.
#random-forest#bagging#ensembles
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How does bagging reduce variance?
- Why does Random Forest sample features at each split, not just once per tree?
- What is the out-of-bag (OOB) score in Random Forests?
- Random forest vs gradient boosting — which do you pick and why?
- Which Random Forest hyperparameters actually matter for tuning?
- Are Random Forest probability estimates well-calibrated?