Behavior Cloning (BC) — when does it work?
mediumAnswer
- Supervised learning: fit to (s, a) pairs from expert demos via MLE.
- Works when: (1) expert data is high quality, (2) test distribution matches expert distribution (no drift), (3) enough coverage.
- Fails: (1) compounding errors under distribution shift (agent drifts, sees unseen states, makes worse mistakes).
- Solutions: DAgger (query expert in new states), residual RL, BC + RL fine-tuning.
Check yourself — multiple choice
- Random
- Supervised fit to expert (s, a); works with high-quality data + coverage + matching distribution; fails via compounding errors under drift → DAgger / BC + RL fine-tune
- Same as RL
- Not real
BC: supervised on expert (s, a); compounding errors under drift; DAgger / BC+RL fix.
#offline-rl#imitation
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- DAgger — Dataset Aggregation for imitation learning.
- Inverse Reinforcement Learning (IRL) — the setup.
- GAIL — Generative Adversarial Imitation Learning.
- Fitted Q-Iteration — batch off-policy Q-learning.
- Decision Transformer — how does it recast RL?
- Trajectory Transformer — variant of Decision Transformer.