24 interview questions on causal inference basics, each answered in full. Free to read, no account needed.
Why doesn't correlation imply causation?
medium- Two variables can move together because A causes B, B causes A, both are caused by a third variable Z (confounder), or by pure coincidence.
- Establishing causation requires either a randomized experiment (breaks confounding by design) or careful causal inference methods (DAGs, matching, instrumental variables, difference-in-differences, regression discontinuity, do-calculus).
What is Simpson's paradox?
hard- A trend that appears in several subgroups can reverse when the groups are combined.
- Classic example: a treatment looks better in each subgroup but worse overall, because group sizes and baseline rates vary.
- It's a warning to always inspect subgroup patterns and think causally — the right decomposition depends on the confounding structure, not on which cut looks prettier.
Users who adopt feature X churn less. Can you say the feature reduces churn?
easy- Not from that association alone, because four mechanisms produce it without the feature doing anything: (1) confounding — engaged users both adopt features and stay, so engagement causes both, (2) reverse causation — users who were already going to stay are the ones who explore features, (3) selection — the population you measured was filtered in a way that creates the link, (4) coincidence, which matters when the sample is small.
- To make the causal claim you need randomization, meaning an experiment that offers the feature to a random subset, or a credible identification strategy: instrumental variables, regression discontinuity, difference-in-differences, or matching under an explicitly stated ignorability assumption.
- Some version of this question appears in every statistics interview.
What is SUTVA?
hard- Stable Unit Treatment Value Assumption: (1) no interference between units — one unit's treatment doesn't affect another's outcome (breaks in networks, marketplaces, viral products), (2) no hidden treatment variants — one 'treatment' is well-defined.
- Violated in social networks (spillovers), two-sided marketplaces (Uber: driver-side changes affect rider outcomes), auctions (competition), any system with global equilibrium effects.
- Fixes: cluster / market-level randomization, switchback experiments.
How do you handle confounders in observational studies?
medium- (1) Include them in regression (XinE[Y∣T,X]) if measured — assumes correct functional form.
- (2) Matching / propensity score matching — trims to overlap region.
- (3) Inverse probability weighting — reweight sample so treated and control look similar.
- (4) Doubly robust: regression + IPW (consistent if either is correct).
- (5) Instrumental variables — bypass unobserved confounders.
- Unmeasured confounders are the fatal weakness — that's why RCTs are the gold standard.
An aggregate trend reverses once you stratify. Which conditioning set is correct?
medium- An association observed at the aggregate level reverses when data is split by a confounder.
- Classic example: Berkeley admissions — men admitted at higher aggregate rate but women admitted at higher rate within every department (women applied to more competitive departments).
- Resolution: pick the right conditioning set based on the causal DAG.
- Blindly conditioning or ignoring a confounder both cause errors.
- Textbook case for why 'always condition on more variables' is wrong.
What is a collider and why is conditioning on it bad?
hard- In a DAG A → C ← B: C is a collider (both A and B point into it).
- A and B are marginally independent, but conditioning on C creates a spurious association.
- Classic example: 'talent and beauty are anticorrelated among Hollywood stars, because being famous (a collider caused by both) has been implicitly selected on'.
- Same mechanism drives selection bias, sample-conditioning artifacts, restaurant-only surveys (studies that condition on 'made it to production' etc.).
What is Pearl's backdoor criterion?
hard- To identify P(Y∣do(T)) from observational data, find a set of variables Z such that: (1) Z blocks all backdoor paths from T to Y (paths starting with an arrow into T), (2) Z contains no descendant of T.
- Then P(Y∣do(T))=Σz P(Y∣T,Z=z) P(Z=z).
- Effectively: condition on all confounders, don't condition on mediators or colliders.
- Foundation of causal DAG-based analysis.
When is the frontdoor criterion useful?
hard- When unmeasured confounders make backdoor impossible, but a fully-mediating variable M exists.
- E.g.
- T → M → Y where all T-Y effect goes through M and no confounder acts on M.
- Frontdoor: P(Y∣do(T))=Σm P(m∣T)⋅Σt' P(Y∣m,t) P(t').
- Classic example: smoking → tar → cancer, if we could measure tar but not the unobserved 'smoking-gene' confounder.
- Rarely applicable in practice but a landmark result in causal inference theory.
What is a propensity score?
hard- e(x)=P(T=1∣X=x).
- Rosenbaum-Rubin: under unconfoundedness, adjusting for e(x) alone is sufficient (dimensionreductionfrom∣X∣to1).
- Uses: (1) matching on e(x), (2) IPW: weight treated by 1/e(x), control by 1/(1-e(x)) → weighted average = ATE.
- (3) Stratification by e(x) quintiles.
- Estimated with logistic regression or ML.
- Never use predicted probabilities of 0 or 1 (violates overlap → wild variance).
Inverse Probability Weighting — how does it work?
hard- Weight each treated unit by 1/e(x) and each control by 1/(1-e(x)) → weighted population is balanced on X.
- Marginal ATE estimator: Σ (TiYi/ei)−Σ ((1−Ti)Yi/(1−ei))/n.
- Consistent under unconfoundedness + overlap + correct e(x) model.
- Weakness: high-variance when e(x) near 0 or 1 → truncate weights or use stabilized IPW / doubly robust estimators.
- Standard in observational causal inference.
Why is a doubly-robust estimator useful?
hard- Combines outcome model μ̂(x) and propensity model ê(x).
- Formula: ATE = E[μ̂(1, X) - μ̂(0, X) + T(Y - μ̂(1, X))/ê(x) - (1-T)(Y - μ̂(0, X))/(1-ê(x))].
- Consistent if EITHER μ̂ OR ê(x) is correctly specified — hence 'doubly robust'.
- AIPW, TMLE are standard implementations.
- Modern default in observational causal inference; also basis of DR-learners / DML with ML nuisance models.
What makes a valid instrumental variable?
hard- Z is a valid IV for T → Y if: (1) Relevance: Z affects T (Cov(Z, T) ≠ 0).
- (2) Exclusion: Z affects Y only through T (no direct Z → Y).
- (3) Exogeneity: Z ⊥ unobserved confounders.
- Estimand: LATE (Local Average Treatment Effect) = Cov(Z, Y) / Cov(Z, T) via 2SLS.
- Classic examples: judge leniency, distance to hospital, weather as an instrument for economic activity.
- Weak instruments (small Cov(Z, T)) → biased 2SLS with wide CIs.
Difference-in-differences — setup and assumption.
hard- Compare change in outcome for treated group vs change for control group before and after intervention.
- Estimand: (Ytreated,post−Ytreated,pre)−(Ycontrol,post−Ycontrol,pre).
- Runs as OLS with unit + time fixed effects and an interaction Treated × Post.
- Key assumption: parallel trends — without treatment, both groups would have evolved similarly.
- Verify with pre-trend tests, event studies, robustness checks.
- Foundational in policy analysis and observational A/B.
How do you defend the parallel-trends assumption?
hard- (1) Plot pre-treatment trends of treated and control — visually parallel?
- (2) Event study: interact treatment indicator with lead / lag indicators; pre-treatment lead coefficients ≈ 0.
- (3) Placebo tests: pretend treatment happened at an earlier fake date — no effect should appear.
- (4) Multiple control groups.
- (5) Synthetic control if only one treated unit.
- Rule: never present DID without explicit pre-trend evidence.
Regression discontinuity design — how does it work?
hard- Treatment assigned by a threshold on a continuous running variable X (e.g. score ≥ 60 → scholarship).
- Compare Y just above vs just below cutoff — near cutoff, individuals are 'as-if random'.
- Sharp RDD: treatment deterministic at cutoff.
- Fuzzy RDD: probability jumps but not to 1 — combine with IV.
- Local linear regression around cutoff + optimal bandwidth (Imbens-Kalyanaraman).
- Foundational in policy evaluation, admission thresholds, credit-score cutoffs.
What is synthetic control?
hard- Construct a weighted combination of untreated units that reproduces the pre-treatment trajectory of the treated unit.
- Weights ≥ 0, sum to 1, chosen to minimize pre-treatment fit error on X and Y.
- Post-treatment gap between treated and synthetic = causal effect estimate.
- Uses: single-treated-unit policy analysis (California smoking ban, German reunification).
- Modern extensions: generalized synthetic control, augmented synthetic control (Ben-Michael), matrix completion methods.
Mediation analysis — direct vs indirect effects.
hard- Path: T → M → Y (with possible direct T → Y).
- Decomposition: total effect = direct effect (T → Y not through M) + indirect effect (T → M → Y).
- Baron-Kenny style regression coefficients or, better, potential-outcomes-based Natural Direct / Indirect Effects (Pearl, Robins).
- Identification requires no unmeasured T-M, T-Y, or M-Y confounders.
- Key application: understanding causal mechanism, not just total effect.
How do you estimate heterogeneous treatment effects (HTE)?
hard- CATE(x)=E[Y(1)−Y(0)∣X=x] varies with covariates.
- Methods: (1) interaction terms in regression (Y ~ T + T×X); (2) causal forests / GRF (Wager & Athey); (3) meta-learners: T-learner (separate models), S-learner (single joint), X-learner (better in imbalanced treatment), R-learner; (4) doubly-robust CATE.
- Uses: personalized recommendations, targeted policies (Uplift modeling in marketing).
- Requires overlap + unconfoundedness (or RCT).
How does Simpson's paradox strike A/B tests?
hard- Test wins overall but loses in every user segment (or vice versa) → aggregate confounded by mid-experiment composition change.
- Common causes: (1) SRM by segment (imbalanced assignment across segments over time), (2) traffic mix drift while experiment runs, (3) new-vs-returning composition shift.
- Fix: (1) fix SRM; (2) analyze weighted by pre-experiment segment shares (post-stratification); (3) run segment-level analyses; (4) if segment-level results all point the same way but opposite to aggregate → trust the segments.
How do you test in a marketplace / network with SUTVA violations?
hard- (1) Cluster randomization (randomize at community / graph-community level so spillovers stay within cluster).
- (2) Switchback experiments (turn treatment on/off over time, randomize periods).
- (3) Ego-cluster tests (treat user + their neighbors together).
- (4) Two-sided experiments (rider-side + driver-side simultaneously in Uber).
- (5) Structural models to extrapolate to full launch.
- Standard practice at Uber, Lyft, Airbnb, DoorDash.
- Naive user-level tests bias effect estimates.
Quantile treatment effect (QTE) vs ATE.
hard- ATE captures mean effect.
- QTE(τ)=FY(1)^(-1)(τ)−FY(0)^(-1)(τ) captures effect on the τ-th quantile — useful when the mean is misleading (heavy tails, revenue metrics dominated by top 1%).
- Example: an A/B test with +0.01ATEbut−1 at the median → most users hurt, a few whales lift the average.
- Estimation: quantile regression, IPW-quantile, causal forests.
- Standard tail-metric analysis at big tech.
Real interview: your model performs great in A/B but flops post-launch. Why?
hard- Common reasons: (1) Selection bias — the A/B population is not the launch population (early adopters, engaged users).
- (2) SUTVA violation — 50% traffic doesn't scale to 100% (marketplace saturation, ad auction dynamics).
- (3) Novelty effect not de-biased.
- (4) Winner's curse — effect regressed to a smaller true value.
- (5) Metric divergence — A/B primary metric doesn't align with long-term OKR.
- (6) Reflex reaction from competitors / operations.
- Debug by: revisiting SUTVA, holdout at 1%, long-term surrogate, and re-measuring at launch.
You cannot randomize. What is the strongest causal claim you can still make?
hard- One conditional on an assumption you state explicitly, which is the honest form of every observational causal claim.
- Begin by drawing the assumed causal structure, because which variables to adjust for is a question about that structure, not about which improve fit; conditioning on a collider or a mediator introduces bias rather than removing it.
- If you can argue that all confounders are measured, adjustment or matching gives an effect estimate under that assumption.
- Stronger designs exploit structure instead: a difference-in-differences comparison if you have pre-period data and a plausible parallel trend, an instrumental variable if something shifts treatment without affecting the outcome directly, or a regression discontinuity if assignment follows a threshold.
- Then test the assumption, with placebo outcomes and pre-trend checks, and report sensitivity to unmeasured confounding.