What are LightGBM's key innovations vs XGBoost?
hardAnswer
- (1) Leaf-wise (best-first) tree growth instead of level-wise — grows the leaf with the largest loss reduction, giving deeper asymmetric trees and often lower loss for the same tree count.
- (2) GOSS (Gradient-based One-Side Sampling): keep all large-gradient samples, subsample small-gradient ones — trains faster with little accuracy loss.
- (3) EFB (Exclusive Feature Bundling): bundle mutually exclusive sparse features into one, cutting feature count in high-dim sparse data.
- (4) Native categorical handling via optimal partitioning.
Check yourself — multiple choice
- It uses linear models
- Leaf-wise growth + GOSS + EFB + native categorical splits
- It removes shrinkage
- It only works on GPUs
Leaf-wise growth, GOSS, EFB, native categoricals — LightGBM's identity.
#lightgbm#gradient-boosting#boosting
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- In one sentence, what is Gradient Boosting?
- What is the role of the learning rate (shrinkage) in GBM?
- What did XGBoost bring on top of vanilla GBM?
- What makes CatBoost different from XGBoost / LightGBM?
- How does early stopping work in gradient boosting and why is it important?
- What are monotonic constraints in XGBoost / LightGBM and when do you use them?