EasyDeepLearn

Bias, variance, consistency of estimators — define.

medium

Answer

  • Bias: E[θ̂] - θ.
  • Variance: Var⁡(θ)\operatorname{Var}({\theta}).
  • MSE⁡  =  bias2  +  variance\operatorname{MSE}\; = \;\mathrm{bias}^{2}\; + \;\mathrm{variance} (bias-variance decomposition).
  • Consistency: θ̂_n → θ in probability as n → ∞.
  • Unbiased ≠ consistent (unbiased with growing variance can be inconsistent); consistent ≠ unbiased (biased with vanishing bias can be consistent).
  • MLE and MAP are typically consistent but biased for small n.
Check yourself — multiple choice
  • Same thing
  • Bias = E[θ̂] - θ; Var = spread; MSE⁡  =  bias2  +  var\operatorname{MSE}\; = \;\mathrm{bias}^{2}\; + \;\mathrm{var}; consistency = θ̂_n → θ as n → ∞ (independent of unbiased)
  • Only MLE unbiased
  • Not defined

Bias⁡  /  var  /  MSE⁡  =  bias2  +  var\operatorname{Bias}\; / \;\mathrm{var}\; / \;\operatorname{MSE}\; = \;\mathrm{bias}^{2}\; + \;\mathrm{var}; consistency = convergence to truth as n → ∞.

#estimation#theory

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions