What is teacher forcing and its main pitfall?

medium

Answer

$During training, feed the ground-truth previous token as input to the decoder at each step (instead of the model's own previous prediction).$
$Speeds training and avoids compounding errors from incorrect early predictions.$
$Pitfall: exposure bias — inference behavior differs from training (model has never seen its own errors).$
$Fixes: scheduled sampling (mix true vs predicted with a schedule) or reinforcement fine-tuning.$

Check yourself — multiple choice

Feed model's own output during training
Feed ground-truth previous token during training — fast but exposure bias at inference
Only used in classification
Removes decoder

Teacher forcing: ground-truth input during training → fast but exposure bias.

#rnn#training#transformers

214 interview questions in this topic.