EasyDeepLearn

Adding a feature raised R2R^2. Does that mean the model improved?

easy

Answer

  • R2=1−SSresSStotR^{2} = 1 - \frac{SS_{\mathrm{res}}}{SS_{\mathrm{tot}}} is the fraction of variance explained; it never decreases when you add a feature, even a useless one.
  • Adjusted R2R^{2} penalizes for the number of features and can decrease when you add a feature that doesn't help.
  • Use adjusted R2R^{2} for model comparison across different feature counts; use R2R^{2} to describe fit at a fixed feature set.
  • Both can be misleading for non-linear models; prefer AIC/BIC or CV.
Check yourself — multiple choice
  • R2R^{2} penalizes complexity; adjusted R2R^{2} does not
  • Adjusted R2R^{2} penalizes for the number of predictors; use it to compare models of different sizes
  • Both are guaranteed to be positive
  • Adjusted R2R^{2} is always higher than R2R^{2}

Adjusted R2R^{2} penalizes added features so you can compare models fairly.

#linear-regression#metrics-regression

Practise Supervised Learning

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions