Are Random Forest probability estimates well-calibrated?
mediumAnswer
- Not always.
- RF probabilities come from averaging tree votes and tend to be pushed away from 0 and 1 (over-cautious).
- A single deep tree gives 0/1 predictions; averaging many gives probabilities that live mostly in a compressed middle range.
- For calibrated probabilities, apply isotonic or Platt calibration on a held-out set (sklearn's CalibratedClassifierCV).
- GBDTs are usually better-calibrated out of the box.
Check yourself — multiple choice
- RF probabilities are always perfectly calibrated
- RF tends to be under-confident; calibrate with Platt or isotonic on a held-out set
- RF cannot output probabilities
- RF calibration is only an issue for regression
RF averages tend to over-shrink toward 0.5 — isotonic/Platt calibration helps.
#random-forest#calibration
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- The 'naive' independence assumption is almost always false. Why does Naive Bayes still work?
- How does temperature scaling calibrate a classifier's probabilities?
- You trained Naive Bayes on balanced data but deploy on data where positives are 1%. What happens and what do you do?
- How does bagging reduce variance?
- Why does Random Forest sample features at each split, not just once per tree?
- What is the out-of-bag (OOB) score in Random Forests?