What criterion does a regression tree use to choose splits?
easyAnswer
- Variance reduction: pick the split that most reduces sum of squared errors of the target within child nodes.
- Equivalently, maximize the total variance of the parent minus the weighted sum of children variances.
- The prediction in each leaf is the mean of the target for training points that fall into it (median for MAE-loss trees).
- Same tree machinery, different impurity function.
Check yourself — multiple choice
- They use Gini impurity
- Variance reduction (MSE) — predict the leaf mean of the target
- They use entropy of y
- They use cross-entropy
Regression trees minimize squared error (or MAE) — leaf predictions are the target mean.
#decision-trees#trees#metrics-regression
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Gini impurity, entropy, and classification error — which do trees actually use?
- How do decision trees handle categorical features?
- Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?
- How does cost-complexity (CCP) pruning work?
- How do decision trees handle missing values?
- Why is a single decision tree considered a 'high-variance' model?