How does stratified randomization help experiments?
hardAnswer
- Randomize within strata (segments) — country, device, user tier.
- Guarantees balance on covariates.
- Reduces variance similar to CUPED.
- Analyzed via stratified estimator or regression with strata fixed effects.
- Especially useful for small experiments where random imbalance dominates.
- Netflix uses per-content-country stratification.
- Downside: more complex bookkeeping; too many strata → sparse cells.
Check yourself — multiple choice
- Random
- Randomize within strata (country/device/tier); guarantees covariate balance + reduces variance; small experiments benefit most; too many strata = sparse cells
- Not helpful
- Not real
Stratified randomization: balance covariates + reduce variance.
#experimentation
Practise MLOps & Data Quality
215 interview questions in this topic.