What makes CatBoost different from XGBoost / LightGBM?
hardAnswer
- (1) Ordered boosting: uses a permutation of the data so that for each example, the model used to estimate its gradient is fit on other examples only — reduces prediction shift / target leakage that plagues target encoding in vanilla GBM.
- (2) Native categorical support via ordered target statistics — handles high-cardinality categoricals without one-hot.
- (3) Symmetric (oblivious) trees — same split at every node of a given depth, giving fast inference and implicit regularization.
- (4) Good defaults — often the strongest 'out-of-the-box' GBDT.
Check yourself — multiple choice
- It's the same as XGBoost with a rebranded name
- Ordered boosting, native categorical support via ordered target stats, symmetric trees, strong defaults
- It's only for classification
- It removes the need for regularization
Ordered boosting + ordered target encoding + oblivious trees = CatBoost's identity.
#catboost#gradient-boosting#boosting#encoding
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- In one sentence, what is Gradient Boosting?
- What is the role of the learning rate (shrinkage) in GBM?
- What did XGBoost bring on top of vanilla GBM?
- What are LightGBM's key innovations vs XGBoost?
- How does early stopping work in gradient boosting and why is it important?
- How do XGBoost, LightGBM, and CatBoost each handle categorical features?