What is a sample ratio mismatch (SRM) alert?

hard

Answer

$Actual traffic split (e.g., 51/49) differs significantly from designed (50/50).$
$Chi-square test detects.$
$Root causes: (1) buggy randomization / hashing.$
$(2) selection bias (bot filtering asymmetric).$
$(3) redirect skew (mobile vs desktop).$
$(4) load imbalance (one variant fails silently, users bounce back).$
$SRM invalidates the whole test — never analyze without fixing first.$
$Standard automated check in any experimentation platform.$

Check yourself — multiple choice

Random
Actual split differs from designed (chi-square test); causes: buggy random / bot filter / redirect / silent fail bounce; invalidates test — fix before analyzing; automated check
Not real
Just noise

SRM: split ratio bug; invalidates test; automated chi-square check.

#experimentation

Practise MLOps & Data Quality

215 interview questions in this topic.

Read the lesson Flashcards Take the quiz

Related questions