What is Optuna's TPE sampler and why is it popular?

hard

Answer

$p(x\; \mid \;y\; > \;y \cdot )$
$p_{\mathrm{good}} / p_{\mathrm{bad}}$
$Advantages: handles categorical + continuous + conditional hyperparameters natively (unlike GP Bayesian opt), scales to hundreds of trials cheaply, and doesn't require a full covariance matrix.$
$Combined with pruners (asynchronous halving), Optuna is the default open-source hyperparameter tool.$

Check yourself — multiple choice

Same as grid search
Models $p(x \mid \mathrm{good})$ and $p(x \mid \mathrm{bad})$ over past trials; picks configs maximizing their ratio
TPE is a type of tree ensemble
TPE only works for two hyperparameters

TPE: model good vs bad trial distributions; sample from ratio ⇒ cheap Bayesian tuning.

#hyperparameter-tuning

215 interview questions in this topic.