EasyDeepLearn

What statistical properties make maximum likelihood the default estimator?

medium

Answer

  • Given data X and a parametric family p(x; θ), MLE picks θ̂ = argmax_θ Σ log p(xi  θ)p(x_{i}\;{\theta}).
  • Equivalently: minimize negative log-likelihood.
  • Properties: consistent (θ̂ → θ*), asymptotically normal (n(θ  −  θ⋅)  →  N(0,  I−1))( \sqrt n({\theta}\; - \;{\theta} \cdot )\; \to \;N(0, \;I^{-1})), asymptotically efficient (achieves Cramér-Rao bound).
  • Foundation of parametric inference and of most ML training objectives (logistic regression, softmax cross-entropy — all MLE).
Check yourself — multiple choice
  • Random guess
  • argmax_θ log-likelihood; consistent, asymptotically normal + efficient; foundation of most ML training losses
  • Only Bayesian
  • Same as MAP

MLE: argmax log-likelihood; consistent + asymptotically normal + efficient.

#estimation#theory

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions