EasyDeepLearn

Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?

medium

Answer

  • Pre-pruning stops the tree from growing during construction, via max⁡depth\operatorname{max}_{\mathrm{depth}}, min⁡samplessplit\operatorname{min}_{\mathrm{samples}}\mathrm{split}, min⁡samplesleaf\operatorname{min}_{\mathrm{samples}}\mathrm{leaf}, or min⁡impuritydecrease\operatorname{min}_{\mathrm{impurity}}\mathrm{decrease}.
  • Fast and simple but you may stop too early (horizon effect).
  • Post-pruning (cost-complexity or reduced-error pruning) grows the tree fully then removes subtrees using a held-out set or a complexity penalty.
  • Usually gives a slightly better bias-variance tradeoff but requires the extra pass.
  • Modern ensembles (RF, GBM) rely on pre-pruning + averaging, which is why deep individual trees are OK in them.
Check yourself — multiple choice
  • Pre-pruning grows the tree fully then trims
  • Pre-pruning stops during construction; post-pruning grows fully then trims subtrees
  • Post-pruning is only for regression trees
  • Both are the same

Pre: stop early via depth/min⁡samples\mathrm{depth} / \operatorname{min}_{\mathrm{samples}}. Post: grow full, then trim (cost-complexity).

#decision-trees#trees#regularization

Practise Supervised Learning

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions