GAIL — Generative Adversarial Imitation Learning.
hardAnswer
- Ho & Ermon 2016.
- Similar to GAN: generator = policy π, discriminator D distinguishes agent trajectories from expert trajectories.
- Train π to fool D via policy gradient (log D as reward).
- Recovers expert behavior without explicit reward.
- Extends to continuous control + high-dim states.
- Foundation of adversarial imitation learning.
- Modern extensions: SQIL, ValueDICE, AdVIL.
Check yourself — multiple choice
- Random
- GAN-like: policy π vs discriminator D distinguishing expert/agent trajectories; π trained to fool D via PG (log D as reward); recovers expert behavior without reward
- Same as BC
- Not real
GAIL: adversarial imitation; π vs D; log D as reward.
#offline-rl#imitation
Practise Reinforcement Learning
214 interview questions in this topic.