How does cost-complexity (CCP) pruning work?
hardAnswer
- Grow the tree fully.
- Define * |T|, where R(T) is the training error, | the number of leaves, and alpha ≥ 0 penalizes complexity.
- For each alpha, the subtree minimizing is unique and nested — you get a full sequence of subtrees indexed by alpha.
- Pick alpha via cross-validation on the sequence.
- Scikit-learn exposes it via _path and the argument.
Check yourself — multiple choice
- Prune every leaf equally
- Trade off training error and tree size via alpha; CV over alpha selects the subtree
- Cost-complexity is only for random forests
- It grows the tree until the training error is zero
CCP: |. Sequence of nested subtrees; CV over α.
#decision-trees#trees#regularization
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?
- Gini impurity, entropy, and classification error — which do trees actually use?
- How do decision trees handle categorical features?
- How do decision trees handle missing values?
- What criterion does a regression tree use to choose splits?
- Why is a single decision tree considered a 'high-variance' model?