What does the IID assumption mean and when is it violated?
mediumAnswer
- IID = samples are independent and identically distributed, drawn from the same population.
- Standard ML theory (CV, generalization bounds, resampling) assumes IID.
- It breaks with time series (temporal correlation), repeated measurements per subject, spatial data (autocorrelation), and streaming data where the distribution drifts.
- Violations lead to over-optimistic error estimates; the fix is to use time-series splits, grouped CV, or drift-aware evaluation.
Check yourself — multiple choice
- IID means data is always Gaussian
- IID means samples are independent and drawn from the same distribution
- IID is only about labels
- IID is guaranteed by shuffling
IID = independent + identically distributed; violated by time series, grouped data, drift.
#theory#validation
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- What is k-fold cross-validation and when do you use stratified or grouped folds?
- Generative vs discriminative classifier — what's the difference?
- What is the curse of dimensionality?
- Parametric vs non-parametric models — what's the difference?
- Why do we prefer simpler models when performance is equal?
- What does the no free lunch theorem say for ML?