IQL — Implicit Q-Learning.
hardAnswer
- Kostrikov et al. 2022.
- Avoids explicit policy constraint or Q pessimism.
- Uses expectile regression to approximate Q(s, a) without evaluating Q at unseen actions — only uses actions from the data.
- Extract policy via advantage-weighted regression on behavior policy.
- Simpler, more stable than BCQ / CQL.
- Often the modern default for offline RL benchmarks in 2023+.
Check yourself — multiple choice
- Same as CQL
- Expectile regression approximates Q using only in-data actions (no OOD eval); extract policy via advantage-weighted regression on behavior; simpler + stable — 2023+ default
- Random
- Not real
IQL: expectile approx max; no OOD queries; AWR policy extraction; 2023+ default.
#offline-rl
Practise Reinforcement Learning
214 interview questions in this topic.