Plasticity loss in deep RL — the problem.
hardAnswer
- Deep RL agents lose the ability to learn new patterns after long training — 'plasticity loss'.
- Nikishin et al. 2022: root cause is weight rank collapse + dead ReLU units accumulating.
- Fixes: (1) periodic reset of top layers (primacy bias reduction).
- (2) Regenerative regularization (RegLoss).
- (3) Layer norm + orthogonal reinitialization.
- (4) Continual learning tricks.
- Critical for long-horizon RL training + continual scenarios.
Check yourself — multiple choice
- Not real
- Agents lose ability to learn new patterns after long training (Nikishin 2022); rank collapse + dead ReLU; fix via periodic layer resets + regenerative regularization + layer norm + orthogonal reinit
- Random
- Only supervised
Plasticity loss: RL loses learning capacity; fix via resets + regularization.
#deep-rl#engineering
Practise Reinforcement Learning
214 interview questions in this topic.