EasyDeepLearn

Derive the VAE ELBO.

hard

Answer

  • log p(x) = log ∫ p(x∣z)p(x \mid z) p(z) dz ≥ Eq(z∣x)[log⁡  p(x∣z)]  −  KL⁡(q(z∣x)  ∣∣  p(z))E_{q(z \mid x)}[\operatorname{log}\;p(x \mid z)]\; - \;\operatorname{KL}(q(z \mid x)\; \mid \mid \;p(z)) (Jensen).
  • ELBO decomposes into reconstruction (log-likelihood of x given z) minus KL to prior.
  • Encoder q_φ(z∣x)(z \mid x) parameterized as Gaussian (μ,  σ2)({\mu}, \;{\sigma}^{2}); decoder p_θ(x∣z)(x \mid z).
  • Reparameterization trick: z = μ + σ ⊙ ε with ε ~ N(0, I) → backprop through sample.
  • Optimize both φ, θ jointly.
Check yourself — multiple choice
  • Random
  • log p(x) ≥ Eq[log⁡  p(x∣z)]  −  KL⁡(q(z∣x)  ∣∣  p(z))E_{q}[\operatorname{log}\;p(x \mid z)]\; - \;\operatorname{KL}(q(z \mid x)\; \mid \mid \;p(z)); reparam z = μ + σε; encoder q_φ + decoder p_θ trained jointly
  • Same as AE
  • Not derivable

VAE ELBO: recon - KL; reparam z = μ + σε for gradient flow.

#representation-learning#deep-learning

Practise Unsupervised Learning

214 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions