Soft voting vs hard voting — which is usually better and why?
easyAnswer
- Hard voting: each model outputs a predicted class; the ensemble takes the majority.
- Simple but throws away probability information.
- Soft voting: average the predicted class probabilities across models; predict the argmax.
- Almost always better because it captures confidence — a hesitant model contributes less than a confident one.
- Requires all models to output calibrated probabilities (calibrate first if not — RF, SVM often aren't out of the box).
- If probabilities aren't available (some SVMs), fall back to hard voting.
Check yourself — multiple choice
- Hard voting is always better
- Soft voting averages probabilities and usually wins — but calibrate base models first
- They give identical results
- Hard voting only works for regression
Soft voting > hard voting when base models are calibrated — uses confidence, not just argmax.
#ensembles
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Random forest vs gradient boosting — which do you pick and why?
- How does bagging reduce variance?
- Why does Random Forest sample features at each split, not just once per tree?
- How do Extra Trees differ from Random Forests?
- In one sentence, what is Gradient Boosting?
- How does stacking work and when does it help?