How do you set thresholds for drift alerts?
hardAnswer
- (1) Compute baseline: 1-2 weeks of production data during known-good period.
- (2) Set threshold: mean + 3σ, or specific PSI > 0.2, or KS p < 0.01.
- (3) Test on historical incidents: does threshold fire correctly?
- (4) Tune sensitivity vs FP rate.
- (5) Different thresholds per feature (some naturally noisier).
- (6) Auto-adjust seasonal: recompute baseline weekly.
- (7) Multi-scale: hourly + daily + weekly windows.
- Fixed thresholds get stale.
Check yourself — multiple choice
- Random
- Baseline 1-2 weeks known-good + threshold (mean+3σ / PSI>0.2 / KS p<0.01) + test on historical + tune FP + per-feature thresholds + seasonal recompute + multi-scale windows; fixed = stale
- Just 0.5
- Not real
Drift thresholds: baseline + tuned + per-feature + seasonal + multi-scale.
#monitoring
Practise MLOps & Data Quality
215 interview questions in this topic.