Catastrophic forgetting in continual RL.
hardAnswer
- When switching between tasks or environments, RL agents forget previously learned skills.
- Root: SGD on new-task data overwrites old-task weights.
- Fixes: (1) EWC (Elastic Weight Consolidation): regularize toward important old-task weights.
- (2) Progressive networks: freeze old columns + add new.
- (3) Rehearsal: replay old experiences.
- (4) Modular networks: separate columns per task.
- (5) Meta-learning shared initialization.
- Ongoing research area.
Check yourself — multiple choice
- Random
- SGD on new task overwrites old-task weights; fix via EWC (regularize important weights) / progressive nets / rehearsal / modular / meta-learning shared init
- Same as forgetting
- Not real
Catastrophic forgetting: EWC / progressive / rehearsal / modular / meta.
#deep-rl#theory
Practise Reinforcement Learning
214 interview questions in this topic.