You use stratified k-fold on a dataset with duplicate customer records — why is it wrong?
mediumAnswer
- Stratified k-fold preserves class proportions but ignores group structure.
- If the same customer appears in both training and validation, the model can 'memorize' that customer via any customer-specific features (device, location patterns, etc.) — leakage that inflates validation performance.
- Fix: use StratifiedGroupKFold (sklearn) or manually group-then-stratify.
- The group is the leakage unit (customer, patient, product) — everything sharing an id must go into the same fold.
Check yourself — multiple choice
- Stratified k-fold prevents all leakage
- Duplicate customers across train/val leak information — use StratifiedGroupKFold
- It's fine as long as you shuffle
- You must drop duplicates
Stratified alone ignores groups; use StratifiedGroupKFold when rows share an entity.
#cv#validation#data-quality
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Leave-One-Out CV — when is it a good idea, and when is it a bad idea?
- What is nested cross-validation and when do you need it?
- Why would you use repeated k-fold instead of standard k-fold?
- How do you set up cross-validation for time series?
- How do you pick the train/validation/test split sizes?
- How does cross-validation change for heavily imbalanced classification?