Novelty vs primacy effects in long experiments.

hard

Answer

$Novelty: users react positively to any change initially, then revert \to early effect inflated, long-run effect smaller.$
$Primacy: existing users hate change initially, then adapt \to early effect deflated, long-run effect larger.$
$Fixes: (1) run experiments long enough (2-4 weeks minimum) to observe stabilization; (2) segment by user tenure (new vs existing); (3) look at the trend of daily effect over time; (4) always plan for ramp-up + steady-state phases in analysis.$

Check yourself — multiple choice

Same thing
Novelty: early effect inflated / long-run smaller; Primacy: opposite; fix with 2-4 week runs, tenure segmentation, daily-effect trend
Not real
Random

Novelty inflates early, primacy deflates early; run 2-4+ weeks; segment by tenure.

#ab-testing

215 interview questions in this topic.