What is a permutation test?

medium

Answer

$Non-parametric test based on the exchangeability principle: shuffle group labels, recompute the test statistic, build a null distribution empirically.$
$Compute p-value = fraction of permutations with statistic ≥ observed.$
$(\mathrm{mean}, \;\mathrm{median}, \;\mathrm{difference}\;\mathrm{in}\;R^{2})$
$Distribution-free, only assumes exchangeability under H0.$
$Slow (usually 10k+ shuffles) but very general.$
$Modern default when you want strong assumption independence.$

Check yourself — multiple choice

Only parametric
Shuffle group labels → empirical null → p = fraction with statistic ≥ observed; distribution-free, only needs exchangeability
Requires normality
Same as bootstrap

Permutation test: shuffle labels → empirical null; distribution-free.

#hypothesis-testing#non-parametric

215 interview questions in this topic.