How do you prove there is no training-serving skew?
hardAnswer
- Compare the actual feature vectors, not the code that computes them.
- Log the features used at inference time, then take the same entities and recompute their features through the training pipeline for the same timestamp, and diff the values.
- Any mismatch is skew, and the common causes are a transformation implemented twice in two languages, a default value that differs, timezone handling, and a training pipeline that saw data updated after the event.
- Do this continuously on a sample rather than once, since skew appears whenever either pipeline changes.
- The structural fix is a single feature computation path shared by both, which is the main reason feature stores exist, and shadow scoring the training pipeline against production is the strongest ongoing check.
Check yourself — multiple choice
- Read both code paths carefully
- Log served feature vectors and diff them against the training pipeline's values for the same entity and timestamp, continuously on a sample — and share one computation path to prevent it
- Compare model accuracy only
- Skew cannot be detected
Value-level diffs on the same entities are the only proof; a shared computation path is the structural fix.
#features#pipeline#data-quality
Practise MLOps & Data Quality
215 interview questions in this topic.
Related questions
- What is point-in-time correctness, and how does violating it look in practice?
- What are common sources of data leakage in an ML pipeline?
- How do you handle missing values in a production pipeline?
- How does imputation cause data leakage?
- How do you handle high-cardinality categorical features?
- How does time-based leakage happen and how do you prevent it?