EasyDeepLearn

What is log-cosh loss and its advantage over Huber?

medium

Answer

  • L(y, ŷ) = log(cosh(y - ŷ)) ≈ 0.5⋅(y−)20.5 \cdot (y - )^{2} for small errors and  ∣y\mathrm{and}\; \mid y-ŷ| - log⁡(2)\operatorname{log}(2) for large — same behavior as Huber but smooth everywhere (all derivatives exist).
  • No δ hyperparameter.
  • Slightly more expensive to compute.
  • Use it when you want Huber's outlier-robustness without tuning δ, and gradient smoothness matters (e.g., second-order optimizers).
Check yourself — multiple choice
  • Discontinuous
  • log⁡(cosh(err))\operatorname{log}(\mathrm{cosh}(\mathrm{err})) — Huber-like behavior but smooth everywhere and no hyperparameter
  • Same as MAE
  • Only for classification

log-cosh: smooth everywhere Huber-alternative, no δ.

#losses

Practise Deep Learning

214 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions