State Jensen's inequality and give an ML example.

hard

Answer

$For a convex function g: E[g(X)] \geq g(E[X]).$
$Concave: E[g(X)] \leq g(E[X]).$
$E[X^{2}]$
$In ML: (1) log-likelihood is concave — Jensen bounds expected log-likelihood \leq log of expected likelihood; foundation of the EM algorithm's E-step.$
$\operatorname{log}(\mathrm{mean}\;\mathrm{prediction})$
$(3) Explains why you can't just average logs then exponentiate to recover a mean.$

Check yourself — multiple choice

Wrong direction
Convex g: E[g(X)] ≥ g(E[X]); concave: opposite; foundation of EM E-step, ensembling analysis
Only for linear g
Not useful

Jensen: E[g(X)] ≥ g(E[X]) for convex g; foundation of EM.

#probability#theory#expectation

215 interview questions in this topic.