How does a bidirectional RNN work and when is it appropriate?
easyAnswer
- Run two RNNs — one left-to-right, one right-to-left — and concatenate their hidden states at each timestep.
- Provides access to both past and future context.
- Appropriate for tagging tasks (POS, NER), speech recognition, classification.
- Not usable for online generation (you need the full sequence).
- Transformers with full attention subsume this by attending both directions in encoder-only setups (BERT).
Check yourself — multiple choice
- Only one direction
- Two RNNs (fwd + bwd) with concatenated hidden states — for tagging/classification, not for generation
- BiRNNs are always causal
- Same as attention
BiRNN: forward + backward RNNs concatenated → past + future context.
#rnn
Practise Deep Learning
214 interview questions in this topic.
Related questions
- Why did transformers replace RNNs for sequence modeling?
- When is orthogonal weight initialization useful?
- What causes exploding gradients and how do you handle them?
- Why is BatchNorm awkward in RNNs?
- Write the vanilla RNN update equation and explain why it struggles with long sequences.
- What is backpropagation through time (BPTT)?