What is Manifold Mixup?

hard

Answer

$Variant of Mixup (Verma et al., 2019): at each step, pick a random hidden layer k and linearly interpolate the hidden representations of two examples at that layer (rather than at the input).$
$Encourages smoother hidden representations, further improves calibration and robustness.$
$Slightly more expensive per step, marginal gains over vanilla Mixup on top-tier vision benchmarks.$

Check yourself — multiple choice

Only interpolate outputs
Interpolate hidden activations at a random layer — smoother internal representations
Same as CutMix
Removes normalization

Manifold Mixup: mixup applied at a random hidden layer → smoother hidden manifolds.

#mixup#regularization

214 interview questions in this topic.