Hindsight Experience Replay (HER) — the trick.
hardAnswer
- Andrychowicz et al. 2017: for a failed episode aiming at goal g, re-label the trajectory as if the final state achieved was the goal.
- Failed attempts become successful demonstrations of reaching that alternative goal.
- Enables learning from sparse-reward goal-conditioned tasks.
- Standard in robotics goal-reaching (pick and place, manipulation).
- Combined with DDPG / SAC for continuous control.
Check yourself — multiple choice
- Same as replay
- For failed episode aiming at g, relabel with achieved final state as if it were the goal → failed attempts = successful demos of alternative goals; robotics goal-reach standard
- Random
- Not real
HER: relabel failed goal-conditioned episodes with achieved states.
#exploration#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.