What is the marginal likelihood (evidence) and why is it hard?

hard

Answer

$p(D\; \mid \;{\theta})$
$Normalizing constant for the posterior.$
$High-dimensional integral \to intractable in closed form for most models.$
$\mathrm{factor}\; = \;p(D\; \mid \;M_{1})\; / \;p(D\; \mid \;M_{2})$
$Computing p(D): bridge sampling, thermodynamic integration, Chib's method, variational lower bounds (ELBO \leq log p(D)).$
$MCMC alone doesn't give p(D) — special methods required.$

Check yourself — multiple choice

Same as likelihood
p(D) = ∫ $p(D \mid {\theta})$ p(θ) dθ; usually intractable; needed for Bayes factors; approximated via bridge sampling / thermodynamic integration / ELBO
Random
Free from MCMC

Marginal likelihood: ∫ $p(D \mid {\theta})$ p(θ)dθ; intractable; special methods needed.

#bayesian

215 interview questions in this topic.