What is scheduled sampling?
mediumAnswer
- Interpolate between teacher forcing and self-generation during training: with probability p feed the true token, with 1-p feed the model's own prediction.
- Decay p over training (start at 1, end near 0).
- Bridges the train/inference gap without full exposure.
- Works OK for RNN seq2seq but doesn't compose easily with parallel transformer training (which processes all positions at once).
Check yourself — multiple choice
- Always uses true tokens
- Mix teacher-forced and self-generated inputs with a probability that decays over training
- Only for classification
- Requires no schedule
Scheduled sampling: gradually replace ground truth with model outputs during training.
#training#rnn
Practise Deep Learning
214 interview questions in this topic.
Related questions
- What causes exploding gradients and how do you handle them?
- When do you use truncated BPTT and what's the trade-off?
- What is teacher forcing and its main pitfall?
- What are vanishing and exploding gradients, and how do you fix them?
- What does Batch Normalization do?
- How does dropout work and when is it applied?