What is backpropagation through time (BPTT)?
easyAnswer
- Unroll the RNN over the sequence length T, treat it as a very deep feed-forward network with shared weights, and apply standard backprop.
- Memory scales linearly with T.
- Truncated BPTT limits the unroll length to k steps to bound memory (each backward pass covers only the most recent k timesteps), at the cost of not learning dependencies longer than k.
Check yourself — multiple choice
- BPTT unrolls only one step
- Unroll RNN over T timesteps as one deep net with shared weights, then standard backprop — memory scales with T
- BPTT skips gradients
- BPTT only works forward
BPTT = unroll + backprop with weight sharing across timesteps.
#rnn#backprop
Practise Deep Learning
214 interview questions in this topic.
Related questions
- Why did transformers replace RNNs for sequence modeling?
- In one sentence, what is backpropagation?
- When is orthogonal weight initialization useful?
- What is a computation graph and why does autograd care?
- Why does the backward pass require more memory than the forward pass?
- What causes exploding gradients and how do you handle them?