How does Monte Carlo estimation work and its convergence rate?
mediumAnswer
- To estimate E[f(X)] where X has a known distribution: draw i.i.d. samples , ..., , average .
- By LLN, the average converges to the expectation.
- Rate: O(1/√n) — / √n, independent of dimension.
- Great for high-dimensional integrals where deterministic quadrature is impossible (curse of dimensionality).
- Variance-reduction tricks: importance sampling, control variates, antithetic variates.
- MCMC (Markov Chain Monte Carlo) uses MC when direct sampling is hard.
Check yourself — multiple choice
- O(1/n)
- Average → E[f(X)] by LLN; convergence O(1/√n), dimension-independent — key advantage over quadrature
- Only for discrete
- O(1)
Monte Carlo: O(1/√n) convergence, dimension-independent — beats quadrature in high-d.
#probability#estimation#monte-carlo
Practise Statistics Fundamentals
215 interview questions in this topic.