EasyDeepLearn

What are the most impactful hyperparameters of a decision tree?

easy

Answer

  • max⁡depth\operatorname{max}_{\mathrm{depth}}: hard limit on depth — the biggest lever. min⁡samplessplit\operatorname{min}_{\mathrm{samples}}\mathrm{split}: minimum samples needed to consider splitting a node. min⁡samplesleaf\operatorname{min}_{\mathrm{samples}}\mathrm{leaf}: minimum in each leaf — controls leaf size and variance. max⁡features\operatorname{max}_{\mathrm{features}}: number of features to consider per split (=all for a tree, sqrt(d) or similar for a forest). min⁡impuritydecrease\operatorname{min}_{\mathrm{impurity}}\mathrm{decrease}: don't split unless impurity drops by at least this much.
  • Together they control the bias-variance tradeoff and inference latency.
Check yourself — multiple choice
  • The activation function
  • max⁡depth\operatorname{max}_{\mathrm{depth}}, min⁡samplessplit\operatorname{min}_{\mathrm{samples}}\mathrm{split}, min⁡samplesleaf\operatorname{min}_{\mathrm{samples}}\mathrm{leaf}, max⁡features\operatorname{max}_{\mathrm{features}}, min⁡impuritydecrease\operatorname{min}_{\mathrm{impurity}}\mathrm{decrease}
  • Learning rate and momentum
  • Number of layers

Depth + leaf/split minimums  +  max⁡features\mathrm{minimums}\; + \;\operatorname{max}_{\mathrm{features}} are the main tree levers.

#decision-trees#hyperparameter-tuning

Practise Supervised Learning

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions