What are common sources of data leakage in an ML pipeline?
mediumAnswer
- (1) Fitting scalers/encoders on the full dataset before splitting.
- (2) Using future information (post-outcome features).
- (3) Grouping leakage: same user in both train and test.
- (4) Duplicated rows across splits.
- (5) Target-encoded features computed on the entire dataset.
- (6) Time-series data shuffled instead of split chronologically.
- Use sklearn Pipelines, grouped/time-aware splits, and thorough exploratory checks.
Check yourself — multiple choice
- Fitting a StandardScaler on the full data (train+test) is safe
- Time-series data should be shuffled before splitting
- Grouped splits prevent user leakage across train and test
- Duplicated rows across splits are not a leakage source
Grouped k-fold keeps all rows of the same entity in one fold.
#data-quality#pipeline
Practise MLOps & Data Quality
215 interview questions in this topic.