How does cross-validation change for heavily imbalanced classification?
mediumAnswer
- Two things: (1) use stratified k-fold to guarantee each fold contains a representative share of the minority class — otherwise some folds might have zero positives, breaking metrics.
- (2) evaluate with imbalance-aware metrics (PR-AUC, F1, recall at fixed precision) — accuracy is meaningless.
- If the minority class is extremely rare (<0.1%), consider repeated stratified splits, or bootstrap with stratification, to get a reliable metric estimate.
Check yourself — multiple choice
- Standard random k-fold is fine
- Use stratified k-fold and imbalance-aware metrics (PR-AUC, F1) — accuracy is misleading
- Only LOOCV
- Drop the minority class
Stratified k-fold + imbalance-aware metrics; consider repeated stratification for rare positives.
#cv#imbalance#validation
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Leave-One-Out CV — when is it a good idea, and when is it a bad idea?
- What is nested cross-validation and when do you need it?
- Why would you use repeated k-fold instead of standard k-fold?
- How do you set up cross-validation for time series?
- How do you pick the train/validation/test split sizes?
- You use stratified k-fold on a dataset with duplicate customer records — why is it wrong?