You detect concept drift in production. What do you do?
mediumAnswer
- (1) Quantify: is it data drift (features change) or concept drift ?
- Different fixes.
- (2) Short-term: fall back to a safer model (simpler, better-calibrated) or raise your decision threshold conservatively.
- (3) Medium-term: retrain on recent data with a rolling window.
- (4) Long-term: instrument monitoring (population stability index, calibration on live labels) to catch it earlier next time and consider online / incremental learning.
- Never silently retrain — always human-review a drifted model before it ships back to production.
Check yourself — multiple choice
- Do nothing — accuracy is a fluke
- Diagnose data vs concept drift → short-term safe fallback → retrain on recent data → improve monitoring
- Immediately retrain on all historical data
- Delete the model
Detect → diagnose → safe fallback → retrain on recent → invest in monitoring.
#interview-scenarios#data-quality
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Your new model has 60% AUC and the baseline had 75%. How do you debug?
- How do you train and evaluate a model when labels arrive weeks after predictions?
- Your model scores AUC 0.92 offline but barely helps in production. What are the usual causes?
- How do you handle noisy labels in a supervised dataset?
- How do you decide how often to retrain a supervised model?
- How do you handle class imbalance in a dataset?