Sampling & central limit theorem — Statistics Fundamentals interview questions

What is sampling bias and how do you mitigate it?

easy

$Sampling bias occurs when the sample is not representative of the target population — some groups over- or under-represented.$
$Causes: convenience sampling, self-selection, survivorship bias, non-response.$
$Mitigations: random sampling with a clear frame, stratified sampling to ensure coverage of subgroups, post-stratification weighting, and comparing sample demographics to population statistics.$

hard

$Usually yes, but for a reason worth stating precisely: the t-test requires the sampling distribution of the mean to be approximately normal, not the data, and the central limit theorem delivers that at large sample sizes even for skewed data.$
$The practical caveat is that convergence is slower the heavier the tail, so with a few hundred observations and extreme outliers the test can still be unreliable.$
$Better options are to cap or winsorize the metric at a high percentile, which reduces variance dramatically at the cost of a slightly different estimand, or to bootstrap the difference in means, which makes no distributional assumption.$
$Do not switch to a rank test without thinking, because the Mann-Whitney test answers a question about stochastic ordering rather than about mean revenue, and mean revenue is what the business cares about.$

#parametric-tests#non-parametric#cltPermalink & quiz →