How do you decide how often to retrain a supervised model?
mediumAnswer
- Measure rather than guess.
- Take the current model, evaluate it on successive time slices of held-out data, and plot how performance decays with the age of the training window.
- The shape of that curve tells you the cadence: if the metric is flat for three months, monthly retraining is waste.
- Weigh it against cost and risk, since every retrain is a chance to ship a regression, so you need automated validation gates.
- Add event-driven triggers on top of the schedule: a drift alarm, a known upstream schema change, or a product launch that shifts the population.
Check yourself — multiple choice
- Retrain daily by default
- Measure the performance decay curve against training-window age, balance retrain risk and cost, and add drift or event triggers
- Only retrain when it breaks
- Retraining cadence is arbitrary
The staleness decay curve sets the cadence, with event-driven triggers layered on.
#interview-scenarios#data-quality
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Your new model has 60% AUC and the baseline had 75%. How do you debug?
- You detect concept drift in production. What do you do?
- How do you train and evaluate a model when labels arrive weeks after predictions?
- Your model scores AUC 0.92 offline but barely helps in production. What are the usual causes?
- How do you handle noisy labels in a supervised dataset?
- How do you handle class imbalance in a dataset?