Inverse Reinforcement Learning (IRL) — the setup.
hardAnswer
- Given expert demonstrations, infer the reward function that makes the expert optimal.
- Ill-posed (many rewards consistent).
- Approaches: (1) Feature-matching: match expected features under learner and expert.
- (2) Max-entropy IRL (Ziebart 2008): among consistent rewards, prefer max-entropy.
- (3) GAIL (Ho & Ermon 2016): GAN-like adversarial training — discriminator judges expert vs learner.
- Foundation of modern imitation learning + robotics.
Check yourself — multiple choice
- Random
- Infer reward function from expert demos; feature-matching / max-entropy IRL (Ziebart) / GAIL (adversarial discriminator judges expert vs learner); ill-posed → regularize
- Same as RL
- Not real
IRL: infer reward from demos; feature-matching / max-ent / GAIL.
#offline-rl#imitation
Practise Reinforcement Learning
214 interview questions in this topic.