How do XGBoost, LightGBM, and CatBoost each handle categorical features?
mediumAnswer
- XGBoost: needs one-hot or ordinal encoding for older versions; native categorical support from 1.5+ using ordered partitioning.
- LightGBM: native support via Fisher-style optimal categorical partitioning; expects category indices as int type or the special 'category' dtype.
- CatBoost: native, using ordered target statistics — no encoding needed and it typically handles high-cardinality categoricals better than the alternatives.
- When in doubt with high-cardinality categoricals: try CatBoost first.
Check yourself — multiple choice
- All three require one-hot encoding
- All three offer native categorical handling now; CatBoost usually wins for high-cardinality
- Only XGBoost has native support
- None supports categorical features
All three have native cat support now; CatBoost's ordered target stats usually wins high-cardinality.
#xgboost#lightgbm#catboost#encoding
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Rule of thumb: when do you pick XGBoost vs LightGBM vs CatBoost?
- What makes CatBoost different from XGBoost / LightGBM?
- How do you handle class imbalance in XGBoost / LightGBM?
- What are monotonic constraints in XGBoost / LightGBM and when do you use them?
- How do decision trees handle categorical features?
- What did XGBoost bring on top of vanilla GBM?