EasyDeepLearn

EM for a GMM — one iteration explicitly.

hard

Answer

  • E-step: for each point xix_{i} and cluster j, γij  =  πj{\gamma}_{\mathrm{ij}}\; = \;{\pi}_{j} N(xi  ∣  μj,  Σj)  /  ΣkN(x_{i}\; \mid \;{\mu}_{j}, \;{\Sigma}_{j})\; / \;{\Sigma}_{k} πk{\pi}_{k} N(xi  ∣  μk,  Σk)N(x_{i}\; \mid \;{\mu}_{k}, \;{\Sigma}_{k}) (responsibilities).
  • M-step: update πj  =  Σi{\pi}_{j}\; = \;{\Sigma}_{i} γij  /  n{\gamma}_{\mathrm{ij}}\; / \;n, μj  =  Σi{\mu}_{j}\; = \;{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}} xi  /  Σix_{i}\; / \;{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}}, Σj  =  Σi{\Sigma}_{j}\; = \;{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}} (xi  −  μj)(x_{i}\; - \;{\mu}_{j})(xi  −  μj)(x_{i}\; - \;{\mu}_{j})' / Σi{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}}.
  • Converges to local max of log-likelihood.
  • Init sensitive → use k-means for starting points.
Check yourself — multiple choice
  • Random
  • E: γij  =  πj{\gamma}_{\mathrm{ij}}\; = \;{\pi}_{j} N(xi∣μj,Σj)  /  ΣkN(x_{i} \mid {\mu}_{j}, {\Sigma}_{j})\; / \;{\Sigma}_{k} …; M: weighted mean/cov update by γ; init-sensitive → warm-start with k-means
  • Same as k-means
  • Only sup.

GMM-EM: E computes responsibilities γ, M does weighted mean/cov updates.

#clustering#density

Practise Unsupervised Learning

214 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions