How do network effects complicate A/B tests?

hard

Answer

$User-level randomization assumes independence between users.$
$Broken by: (1) social networks: friend sees new feed, tells friend in control.$
$(2) marketplace: prices in one variant affect supply / demand for other.$
$(3) messaging: control receives messages influenced by treatment.$
$Fixes: (1) cluster randomization (whole regions / networks).$
$(2) switchback experiments (whole population toggles by hour / day).$
$(3) time-based split.$
$Standard problem at Uber (drivers vs riders), Facebook (social graph).$

Check yourself — multiple choice

Random
User-level randomization assumes independence — broken by social/marketplace/messaging spillover; fix via cluster randomization (regions) or switchback (whole pop toggles by hour/day)
Not real
Ignore

Network effects: spillover breaks independence; cluster or switchback.

#experimentation

215 interview questions in this topic.