How does gradient boosting handle missing values without imputation?
mediumAnswer
- For each split, the algorithm considers routing missing values to the left or the right child and picks the direction that yields the largest gain on the training data.
- This 'learned default direction' is stored per split and used at prediction time.
- Effect: NaNs are treated as first-class citizens; the tree can even learn that 'missingness' itself is informative.
- XGBoost, LightGBM and CatBoost all implement this — you can pass NaNs directly.
Check yourself — multiple choice
- GBDTs always require imputation
- Each split learns a default direction for missing values from training data
- Missing values are dropped
- They are replaced by the mean
Missing values are routed via a learned default direction per split.
#gradient-boosting#missing-data
Practise Supervised Learning
215 interview questions in this topic.