What is the one-cycle policy?
mediumAnswer
- Smith's 'super-convergence' recipe: linearly ramp LR up from to over the first ~30% of training, then linearly decay back down.
- Simultaneously modulate momentum in the opposite direction (high when LR is low, low when LR is high).
- Enables very fast training with large max LR — regularizes via the 'LR-as-annealing-temperature' effect.
Check yourself — multiple choice
- Constant LR throughout
- Ramp LR up then down over one epoch cycle; momentum modulates inversely — very fast convergence
- Same as cosine decay
- Only used for RNNs
One-cycle: LR triangle with inverse momentum → super-convergence.
#schedules#learning-rate#training
Practise Deep Learning
214 interview questions in this topic.