Platt scaling vs isotonic regression for probability calibration — how do you choose?
mediumAnswer
- Platt: fit a logistic regression on the classifier's scores → sigmoid recalibration.
- Parametric (2 params), works well with small calibration sets, assumes a sigmoidal miscalibration shape.
- Isotonic: fit a non-decreasing step function via pool-adjacent-violators.
- Non-parametric, more flexible, but needs more data (~1000+ examples) or it overfits.
- Rule of thumb: Platt for small calibration sets or nearly-calibrated models; isotonic for larger data or arbitrary miscalibration curves.
Check yourself — multiple choice
- Isotonic is always better
- Platt = sigmoid fit (small data, well-shaped miscalibration); isotonic = step function (more data, more flexible)
- They give identical results
- Both require the training set
Platt = parametric sigmoid (small data). Isotonic = non-parametric, more data-hungry.
#calibration
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- The 'naive' independence assumption is almost always false. Why does Naive Bayes still work?
- How does temperature scaling calibrate a classifier's probabilities?
- You trained Naive Bayes on balanced data but deploy on data where positives are 1%. What happens and what do you do?
- Are Random Forest probability estimates well-calibrated?
- What is log loss (cross-entropy) and what does a specific value mean?
- What is the Brier score and how does it compare to log loss?