Leave-One-Out CV — when is it a good idea, and when is it a bad idea?
mediumAnswer
- LOOCV uses n-1 samples for training and 1 for validation, repeated n times.
- Pros: nearly unbiased estimate of generalization error, uses maximum training data per fold.
- Cons: (1) computationally brutal for anything but linear models with closed-form updates; (2) high variance of the estimate because folds are almost identical; (3) doesn't work with grouped data (each row is its own fold).
- Prefer 5- or 10-fold CV for most practical cases.
- LOOCV is only clearly best when n is very small.
Check yourself — multiple choice
- LOOCV is always the best CV strategy
- Almost-unbiased but expensive and high-variance; prefer 5- or 10-fold in practice
- LOOCV is only used for image data
- LOOCV doesn't require a validation set
LOOCV: unbiased but expensive and high-variance. 5/10-fold is the practical default.
#cv#validation
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- What is nested cross-validation and when do you need it?
- Why would you use repeated k-fold instead of standard k-fold?
- How do you set up cross-validation for time series?
- How do you pick the train/validation/test split sizes?
- How does cross-validation change for heavily imbalanced classification?
- You use stratified k-fold on a dataset with duplicate customer records — why is it wrong?