In one sentence, what is Gradient Boosting?

medium

Answer

$Fit a sequence of weak learners (typically shallow trees), where each new learner is trained to predict the negative gradient of the loss w.r.t. the current ensemble's prediction — additive stagewise optimization in function space.$
$Final prediction is the sum of the learners' outputs (each scaled by a learning rate).$
$Works for any differentiable loss: squared error for regression, log-loss for classification, quantile loss, ranking losses, etc.$

Check yourself — multiple choice

Boosting averages independent trees
Sequentially fit weak learners to the negative gradient of the loss — additive function-space optimization
It's the same as bagging with a different name
It only works for classification

Additive stagewise fitting to the negative gradient of the loss (function-space gradient descent).

#gradient-boosting#boosting#ensembles

215 interview questions in this topic.