Decision Transformer — how does it recast RL?
hardAnswer
- Chen et al. 2021: treat RL as sequence modeling.
- Given (return-to-go, state, action) sequence, predict next action autoregressively.
- Trained on offline trajectories via cross-entropy.
- At inference, condition on desired return-to-go.
- Fully removes TD learning + bootstrapping — no value functions.
- Competitive with offline RL on D4RL.
- Foundation of return-conditioned RL and modern LLM-based agents.
Check yourself — multiple choice
- Same as PPO
- Sequence modeling of (return-to-go, state, action); autoregressive next-action prediction; no TD / bootstrapping; condition on desired return at inference; foundation of return-conditioned RL
- Random
- Not real
Decision Transformer: RL as sequence modeling of (RTG, s, a); no TD.
#offline-rl#theory
Practise Reinforcement Learning
214 interview questions in this topic.