What is CUPED and why use it?
hardAnswer
- Controlled Using Pre-Experiment Data.
- Adjust post-experiment metric Y by subtracting β × (X - E[X]) where X is pre-experiment metric for same user, β = Cov(X,Y)/Var(X).
- Reduces variance in metric → smaller sample size for same statistical power (often 30-50% reduction).
- Standard at Microsoft / Netflix / Booking.
- Prerequisite: pre-experiment data available per user.
- Extended: ML-CUPED uses ML model to predict Y from many pre-features.
Check yourself — multiple choice
- Random
- Controlled Using Pre-Experiment Data: adjust Y - β(X-E[X]) with β = Cov(X,Y)/Var(X); reduces variance 30-50% → smaller sample size; Microsoft/Netflix/Booking standard
- Same as CV
- Not real
CUPED: adjust metric by pre-exp covariate → 30-50% variance reduction.
#experimentation
Practise MLOps & Data Quality
215 interview questions in this topic.