EasyDeepLearn

Which Random Forest hyperparameters actually matter for tuning?

medium

Answer

  • nestimatorsn_{\mathrm{estimators}}: 'more is better' until returns diminish — 200-500 is a solid default. max⁡features\operatorname{max}_{\mathrm{features}}: the main lever for correlation between trees; try sqrt(d), log2(d), or 0.3-0.5*d. max⁡depth  /  min⁡samplesleaf\operatorname{max}_{\mathrm{depth}}\; / \;\operatorname{min}_{\mathrm{samples}}\mathrm{leaf}: control tree size; deeper trees favor bias-reduction, larger min⁡samplesleaf\operatorname{min}_{\mathrm{samples}}\mathrm{leaf} reduces variance. classweight\mathrm{class}_{\mathrm{weight}}='balanced' for imbalanced data. njobsn_{\mathrm{jobs}}=-1 for speed.
  • Skip fiddling with min⁡samplessplit\operatorname{min}_{\mathrm{samples}}\mathrm{split} — min⁡samplesleaf\operatorname{min}_{\mathrm{samples}}\mathrm{leaf} is enough.
Check yourself — multiple choice
  • Only nestimatorsn_{\mathrm{estimators}} matters
  • nestimatorsn_{\mathrm{estimators}}, max⁡features\operatorname{max}_{\mathrm{features}}, max⁡depth/min⁡samplesleaf\operatorname{max}_{\mathrm{depth}} / \operatorname{min}_{\mathrm{samples}}\mathrm{leaf}, classweight\mathrm{class}_{\mathrm{weight}}
  • The activation function
  • Learning rate

nestimators  +  max⁡features  +  treen_{\mathrm{estimators}}\; + \;\operatorname{max}_{\mathrm{features}}\; + \;\mathrm{tree} size  +  classweight\mathrm{size}\; + \;\mathrm{class}_{\mathrm{weight}} cover most of the tuning.

#random-forest#hyperparameter-tuning

Practise Supervised Learning

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions