How do decision trees handle categorical features?
mediumAnswer
- Native support depends on the library.
- Classic CART considers only binary splits: partition category set A vs the rest, or the exhaustive best partition . scikit-learn does *not* support categoricals natively — you must one-hot encode.
- LightGBM has native categorical splits via Fisher's optimal partitioning.
- CatBoost uses ordered target encoding.
- XGBoost added native categorical support in 1.5+.
- Native handling is usually stronger and faster than one-hot for high-cardinality categoricals.
Check yourself — multiple choice
- All tree libraries handle categoricals identically
- Native support varies: LightGBM/CatBoost/newer XGBoost handle them directly; scikit-learn needs one-hot
- Trees can only use numeric features
- One-hot encoding is always the best approach for trees
LightGBM/CatBoost/XGBoost ≥1.5 have native cat support; scikit-learn requires encoding.
#decision-trees#trees#features#encoding
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Gini impurity, entropy, and classification error — which do trees actually use?
- Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?
- How does cost-complexity (CCP) pruning work?
- How do decision trees handle missing values?
- What criterion does a regression tree use to choose splits?
- Why is a single decision tree considered a 'high-variance' model?