EasyDeepLearn

Your model reports R2  =  0.92R^{2}\; = \;0.92. What can go wrong with celebrating that?

medium

Answer

  • Plenty.
  • R2R^{2} rises mechanically with every predictor you add, whether or not it helps, so a high value on training data says nothing about generalization and adjusted R2R^{2} only partially compensates.
  • It is scale-free but not comparable across datasets, because it depends on the variance of the target: the same model achieves a high value on a heterogeneous population and a low one on a narrow slice.
  • On time series with a trend, a high R2R^{2} is nearly automatic and often means only that both series drift, which is why differencing before evaluating matters.
  • It also says nothing about whether the residuals are well behaved, so a value of 0.92 is compatible with obvious structure the model missed.
  • Report out-of-sample error in the units of the problem alongside it, and always look at the residual plot.
Check yourself — multiple choice
  • Nothing, it is a strong result
  • R2R^{2} rises with any added predictor, depends on the target's variance so it is not comparable across datasets, is nearly automatic for trending series, and ignores residual structure
  • It proves causation
  • It equals prediction accuracy

R2R^{2} is inflatable, dataset dependent and blind to residual structure, so report out-of-sample error too.

#regression#assumptions

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions