What is Huber loss and why is it a compromise between MAE and MSE?
mediumAnswer
- Huber loss is quadratic | ≤ delta (like MSE) and linear beyond delta (like MAE), joined smoothly at delta.
- Combines MSE's differentiability near zero (good gradients for optimization) with MAE's robustness to outliers (linear tail doesn't blow up on rare huge errors). delta acts as an outlier threshold — typical values are or set via CV.
- Used in robust regression, Huber-loss neural nets, and reinforcement learning (Huber for value updates).
Check yourself — multiple choice
- Huber is always the same as MSE
- Quadratic near zero, linear beyond δ — MSE's gradients + MAE's outlier robustness
- Huber only works for classification
- Huber replaces cross-validation
Huber = quadratic small errors + linear big errors ⇒ smooth and robust.
#metrics-regression#regression
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- What does the quantile (pinball) loss measure?
- When would you use quantile regression instead of predicting the mean?
- Adding a feature raised R^2. Does that mean the model improved?
- Can R^2 be negative? What does that mean?
- When is quantile regression more useful than mean (OLS) regression?
- What criterion does a regression tree use to choose splits?