Users who adopt feature X churn less. Can you say the feature reduces churn?
easyAnswer
- Not from that association alone, because four mechanisms produce it without the feature doing anything: (1) confounding — engaged users both adopt features and stay, so engagement causes both, (2) reverse causation — users who were already going to stay are the ones who explore features, (3) selection — the population you measured was filtered in a way that creates the link, (4) coincidence, which matters when the sample is small.
- To make the causal claim you need randomization, meaning an experiment that offers the feature to a random subset, or a credible identification strategy: instrumental variables, regression discontinuity, difference-in-differences, or matching under an explicitly stated ignorability assumption.
- Some version of this question appears in every statistics interview.
Check yourself — multiple choice
- Always implies
- Confounding / reverse / selection / coincidence break the implication; need RCT or explicit causal identification (IV / DID / RDD / matching)
- Only for regression
- Random
Confounding / reverse / selection break correlation → causation; need RCT or ID strategy.
#causal-inference
Practise Statistics Fundamentals
215 interview questions in this topic.