MSE vs MAE for regression — when is each preferred?
easyAnswer
- MSE (L2): differentiable everywhere, penalizes large errors quadratically → sensitive to outliers, gradients are proportional to error size.
- MAE (L1): penalizes linearly → robust to outliers but not differentiable at 0.
- Choose MSE when errors are Gaussian-ish and outliers are rare; choose MAE when the target has heavy tails or you want the median rather than the mean.
- Huber loss is the practical compromise.
Check yourself — multiple choice
- MSE is robust to outliers
- MSE penalizes large errors more (Gaussian-friendly, outlier-sensitive); MAE is robust but non-smooth at 0
- MSE and MAE are identical
- MAE has no gradient anywhere
MSE ↔ Gaussian noise / mean; MAE ↔ Laplace noise / median; Huber compromises.
#losses#fundamentals
Practise Deep Learning
214 interview questions in this topic.
Related questions
- Why compute softmax + cross-entropy jointly via log-sum-exp?
- How should the output head be designed for a regression task with a strictly positive target?
- How is the output head different for multi-label vs multi-class classification?
- ReLU vs sigmoid vs GELU — when do you use each?
- In one sentence, what is backpropagation?
- State the universal approximation theorem in one sentence and its practical caveat.