Is a useful metric for non-linear models (RF, GBM, neural nets)?
mediumAnswer
- It's a valid summary but has caveats: (1) is not scale-free per dataset — it depends on — so comparing across datasets is misleading; (2) it's easy to inflate by adding features and can overstate goodness for wiggly models on small data; (3) it doesn't reflect calibration or heteroscedastic errors.
- Prefer RMSE / MAE / quantile loss for absolute quality, MASE for time series comparison, and Bayesian criteria (AIC, BIC) for model selection.
Check yourself — multiple choice
- is meaningless for non-linear models
- Valid but limited — prefer RMSE/MAE/quantile loss for absolute quality; MASE for cross-dataset comparison
- is always the best regression metric
- only works for linear regression
is a valid summary but data-dependent; complement with RMSE/MAE/MASE.
#metrics-regression
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Adding a feature raised R^2. Does that mean the model improved?
- Can R^2 be negative? What does that mean?
- When is quantile regression more useful than mean (OLS) regression?
- What criterion does a regression tree use to choose splits?
- RMSE vs MAE — how do you pick?
- What is Huber loss and why is it a compromise between MAE and MSE?