EasyDeepLearn

MLE for a Bernoulli(p) — derive.

medium

Answer

  • Likelihood: L(p) = p^Σxi  ⋅  (1−p){\Sigma}x_{i}\; \cdot \;(1 - p)^(n  −  Σxi)(n\; - \;{\Sigma}x_{i}).
  • Log-lik: ℓ(p)  =  Σxi  ⋅  log⁡(p)\; = \;{\Sigma}x_{i}\; \cdot \;\operatorname{log} p  +  (n  −  Σxi)  ⋅  log⁡(1−p)p\; + \;(n\; - \;{\Sigma}x_{i})\; \cdot \;\operatorname{log}(1 - p). ∂ℓ/∂p  =  Σxi  /  p  −  (n  −  Σxi)/(1−p)  =  0p\; = \;{\Sigma}x_{i}\; / \;p\; - \;(n\; - \;{\Sigma}x_{i}) / (1 - p)\; = \;0 → p̂ = Σxi  /  n  =  X{\Sigma}x_{i}\; / \;n\; = \;X̄.
  • So the MLE of a Bernoulli parameter is just the sample proportion.
  • Also happens to be unbiased and minimum-variance (via CR bound).
  • Classic textbook derivation asked in interviews.
Check yourself — multiple choice
  • Random
  • MLE of Bernoulli p is p̂ = X̄ (sample proportion) — unbiased + minimum variance (Cramér-Rao)
  • Not derivable
  • Same as median

MLE Bernoulli: p̂ = X̄; unbiased, minimum variance.

#estimation#theory

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions