How do you test in a marketplace / network with SUTVA violations?

hard

Answer

$(1) Cluster randomization (randomize at community / graph-community level so spillovers stay within cluster).$
$(2) Switchback experiments (turn treatment on/off over time, randomize periods).$
$(3) Ego-cluster tests (treat user + their neighbors together).$
$(4) Two-sided experiments (rider-side + driver-side simultaneously in Uber).$
$(5) Structural models to extrapolate to full launch.$
$Standard practice at Uber, Lyft, Airbnb, DoorDash.$
$Naive user-level tests bias effect estimates.$

Check yourself — multiple choice

Ignore SUTVA
Cluster / switchback / ego-cluster / two-sided / structural — user-level A/B biased in marketplaces & networks
Only bigger n
Random

Network / marketplace A/B: cluster / switchback / ego / two-sided / structural.

#ab-testing#causal-inference

215 interview questions in this topic.