How does Metropolis-Hastings work?

hard

Answer

$MCMC: build a Markov chain whose stationary distribution is the posterior.$
$q({\theta}\; \mid \;{\theta})$
${\alpha}\; = \;\operatorname{min}(1, \;[p({\theta}\; \mid \;D)\;q({\theta}\; \mid \;{\theta})]\; / \;[p({\theta}\; \mid \;D)\;q({\theta}\; \mid \;{\theta})])$
${\alpha}\; = \;\operatorname{min}(1, \;p({\theta}\; \mid \;D)\; / \;p({\theta}\; \mid \;D))$
$Only needs the posterior up to a constant \to dodges intractable evidence.$
$Weaknesses: slow mixing in high dims; sensitive to proposal scale.$

Check yourself — multiple choice

Random
Propose θ' ~ $q({\theta} \mid {\theta})$ , accept with ${\alpha}\; = \;\operatorname{min}(1, \;[p({\theta} \mid D)q({\theta} \mid {\theta})]\; / \;[p({\theta} \mid D)q({\theta} \mid {\theta})])$ ; needs posterior up to constant only
Same as Gibbs
Not MCMC

Metropolis-Hastings: propose + accept with α; needs unnormalized posterior only.

#bayesian#mcmc

215 interview questions in this topic.