DAgger — Dataset Aggregation for imitation learning.
mediumAnswer
- Ross et al. 2011.
- Iterative BC: (1) train π on expert data, (2) roll out π in env, (3) query expert for correct action at every state visited, (4) add (state, expert-action) to dataset, (5) retrain.
- Fixes BC's compounding errors by getting expert labels on the agent's own state distribution.
- Requires access to expert during training.
- Foundation of interactive imitation learning.
Check yourself — multiple choice
- Same as BC
- Iteratively: train π on data, roll out π, query expert for correct actions on visited states, add + retrain; fixes BC compounding errors; requires interactive expert access
- Random
- Not real
DAgger: iterative expert-labeling on agent's own states; fixes BC drift.
#offline-rl#imitation
Practise Reinforcement Learning
214 interview questions in this topic.