How long should you retain ML prediction logs?
mediumAnswer
- Depends on: (1) regulatory (banking / medical: 7-10 years).
- (2) debug window (recent logs hot 30 days; older cold storage).
- (3) retraining data (labeled predictions kept indefinitely).
- (4) storage cost balance.
- Typical: (a) hot: 30 days (Elasticsearch / Loki).
- (b) warm: 6-12 months (Parquet on S3).
- (c) cold: archive to Glacier for 5-10 years compressed.
- Aggregate metrics (Prometheus) forever at reduced resolution.
Check yourself — multiple choice
- Random
- Regulatory 7-10 years + hot 30 days (ES/Loki) + warm 6-12 months (Parquet S3) + cold archive Glacier 5-10y compressed; aggregate metrics forever reduced resolution; label data indefinite
- One week
- Forever
Log retention: hot/warm/cold tiers + regulatory + aggregated forever.
#observability
Practise MLOps & Data Quality
215 interview questions in this topic.