Why is diversity between base models more important than their individual accuracy in an ensemble?
mediumAnswer
- If two 95%-accurate models make exactly the same mistakes, averaging them still gives 95% — no improvement.
- If two 90%-accurate models make *different* mistakes, averaging can push accuracy well above either.
- Ensembles reduce variance from errors that don't correlate across models.
- Diversify by using: different model families (linear + tree + KNN), different feature subsets, different random seeds, or different subsamples.
- Bagging and random forests are engineered around this idea.
- In competitions, blending diverse models is often more valuable than optimizing a single one.
Check yourself — multiple choice
- All models should be identical
- Ensembles reduce variance from uncorrelated errors — diverse models beat homogenous strong ones
- Diversity hurts ensembles
- Only bagging matters
Uncorrelated errors ⇒ variance reduction. Diversity drives ensemble gains.
#ensembles
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Random forest vs gradient boosting — which do you pick and why?
- How does bagging reduce variance?
- Why does Random Forest sample features at each split, not just once per tree?
- How do Extra Trees differ from Random Forests?
- In one sentence, what is Gradient Boosting?
- How does stacking work and when does it help?