EasyDeepLearn

What is the bias-variance tradeoff?

easy

Answer

  • Bias is error from wrong assumptions (underfitting).
  • Variance is error from sensitivity to training data (overfitting).
  • Total generalization error decomposes as E[(y−f^(x))2]=Bias[f^(x)]2+Var[f^(x)]+σ2\mathbb{E}\big[(y - \hat{f}(x))^{2}\big] = \mathrm{Bias}\big[\hat{f}(x)\big]^{2} + \mathrm{Var}\big[\hat{f}(x)\big] + \sigma^{2} where σ2\sigma^{2} is the irreducible noise.
  • You reduce bias with more complex models or better features, and reduce variance with more data, regularization, or ensembling.
  • The tradeoff means you can rarely minimize both at once.
Check yourself — multiple choice
  • High bias means the model overfits the training data
  • High variance means the model is too simple
  • High bias means the model underfits; high variance means it overfits
  • Bias and variance are the same thing measured differently

High bias  →  underfit\mathrm{bias}\; \to \;\mathrm{underfit} (too simple). High variance  →  overfit\mathrm{variance}\; \to \;\mathrm{overfit} (too sensitive to the training set).

#fundamentals#evaluation

Practise Supervised Learning

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions