EasyDeepLearn

Variational inference vs MCMC.

hard

Answer

  • VI: approximate posterior p(θ  ∣  D)p({\theta}\; \mid \;D) with a simpler family q_φ(θ) (e.g. mean-field Gaussian) by maximizing ELBO  =  Eq[log⁡  p(D,  θ)]  −  Eq[log⁡  q(θ)]\mathrm{ELBO}\; = \;E_{q}[\operatorname{log}\;p(D, \;{\theta})]\; - \;E_{q}[\operatorname{log}\;q({\theta})].
  • Advantages: much faster, scales to big data (SVI, mini-batches), differentiable → fits on GPU.
  • Weaknesses: biased (limited by family); mean-field VI under-covers uncertainty and drops correlations.
  • Use for large-scale problems where speed > accuracy.
  • Modern: normalizing flows for richer q.
Check yourself — multiple choice
  • Random
  • VI: fit simpler q_φ(θ) by maximizing ELBO → fast + scalable + differentiable; biased (limited family, mean-field under-covers); MCMC accurate but slow
  • Same as MCMC
  • Only for regression

VI: maximize ELBO in family q_φ; fast + biased vs MCMC accurate + slow.

#bayesian#mcmc

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions