Intrinsic Curiosity Module (ICM) — Pathak et al.
hardAnswer
- Learn forward model f(s, a) → predicted and inverse model → .
- Encode state via φ trained by inverse loss (predicts a from state pair) → learns features that matter for control.
- Intrinsic reward = ||f(s, a) - φ(s')||² = model prediction error → curiosity toward unpredictable transitions.
- Inverse-model feature learning is key for filtering out irrelevant randomness ('noisy TV' problem).
Check yourself — multiple choice
- Random
- Forward + inverse models on encoded features φ; inverse loss learns control-relevant features; intrinsic reward = forward prediction error → curiosity toward unpredictable transitions
- Same as RND
- Not real
ICM: forward + inverse models on learned φ; intrinsic reward = prediction error.
#exploration#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.