Your augmentation pipeline made validation accuracy worse. What went wrong?
mediumAnswer
- Almost always a distribution mismatch: the augmentation created inputs unlike anything at test time, so the model spent capacity on a harder problem than the one it is graded on.
- Classic examples are horizontal flips on digits or text, aggressive colour jitter when colour is the label signal, and rotations on medical scans with a canonical orientation.
- Second possibility is that augmentation was accidentally applied to the validation set, which changes the measurement rather than the model.
- Third, the augmentation is fine but too strong for the training budget, since heavier augmentation needs more epochs to pay off.
- The diagnostic is to look at augmented samples with your own eyes and ask whether the label is still correct and the image still plausible.
Check yourself — multiple choice
- Augmentation always helps
- Usually a train/test distribution mismatch — label-destroying or implausible transforms — or augmentation leaking into validation, or too strong for the epoch budget
- Only the learning rate matters
- Validation sets are unreliable
Augmentation must preserve the label and stay within the test distribution, and must never touch validation.
#augmentation#training
Practise Deep Learning
214 interview questions in this topic.
Related questions
- What is a solid default augmentation recipe for training a modern vision model?
- Your validation loss is lower than your training loss. Is something broken?
- What are vanishing and exploding gradients, and how do you fix them?
- What does Batch Normalization do?
- How does dropout work and when is it applied?
- Why use a learning-rate schedule (warmup + cosine decay)?