When would you use the bootstrap for model evaluation instead of k-fold?
hardAnswer
- Bootstrap resamples the training set with replacement to build many pseudo-datasets.
- Fit the model on each and evaluate on the out-of-bootstrap (OOB) samples (~37% left out).
- Advantages: gives you a full distribution over the metric — confidence intervals come free, useful for statistical comparisons.
- Weaknesses: overlap across bootstrap samples means each replicate is highly correlated; slightly optimistic estimate.
- Prefer k-fold for model selection; bootstrap when you specifically need CIs on the metric.
Check yourself — multiple choice
- Bootstrap always replaces k-fold
- For confidence intervals on metrics — but samples are correlated so estimates can be optimistic
- Only for regression
- Bootstrap doesn't sample with replacement
Bootstrap gives CIs on metrics; correlations make it slightly optimistic.
#cv#validation
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Leave-One-Out CV — when is it a good idea, and when is it a bad idea?
- What is nested cross-validation and when do you need it?
- Why would you use repeated k-fold instead of standard k-fold?
- How do you set up cross-validation for time series?
- How do you pick the train/validation/test split sizes?
- How does cross-validation change for heavily imbalanced classification?