Why do decision trees (and boosted trees) fail to extrapolate?
mediumAnswer
- A tree predicts by returning the leaf value; leaves are bounded by the observed training range.
- If a test feature is beyond the training range, the tree returns whatever leaf value corresponds to the closest region — no extrapolation.
- This is fine on tabular data with stable distributions, but disastrous for time series with trend, or physics-style problems where extrapolation is the point.
- Use linear models, splines, or neural nets when extrapolation matters.
Check yourself — multiple choice
- They extrapolate linearly beyond training data
- Predictions are bounded by leaf values learned on training — trees can't extrapolate
- They extrapolate better than linear models
- They can only extrapolate for classification
Leaf values come from training points only — no extrapolation beyond the training range.
#decision-trees#trees#theory
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why is a single decision tree considered a 'high-variance' model?
- Gini impurity, entropy, and classification error — which do trees actually use?
- How do decision trees handle categorical features?
- Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?
- How does cost-complexity (CCP) pruning work?
- How do decision trees handle missing values?