EasyDeepLearn

Explain the EM algorithm.

hard

Answer

  • For MLE with latent variables (Z): (E) compute q(Z)  =  p(Z  ∣  X,  θold)q(Z)\; = \;p(Z\; \mid \;X, \;{\theta}_{\mathrm{old}}); (M) update θnew  =  argmax⁡{\theta}_{\mathrm{new}}\; = \;\operatorname{argmax}_θ Eq[log⁡  p(X,  Z  ∣  θ)]E_{q}[\operatorname{log}\;p(X, \;Z\; \mid \;{\theta})].
  • Guaranteed to increase the log-likelihood at each iteration, converges to a local max.
  • Uses: Gaussian mixture models (soft clustering), HMMs (Baum-Welch), missing data imputation.
  • Sensitive to init → try multiple starts.
Check yourself — multiple choice
  • Only linear
  • E: posterior of latent Z; M: maximize expected log-likelihood → GMMs, HMMs, missing data; monotone likelihood improvement, local optima
  • Same as gradient descent
  • Random

EM: alternates E and M; monotone likelihood; GMMs / HMMs / missing data.

#estimation#bayesian

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions