Hierarchical RL — why and how?
hardAnswer
- Decompose long-horizon tasks into (1) high-level policy over sub-goals or options, (2) low-level policy executing skills.
- Advantages: temporal abstraction (reason over many steps at high level), sample efficiency, transfer of skills.
- Frameworks: Options (Sutton, Precup, Singh), FeUdal Networks (Vezhnevets), HIRO (goal-conditioned low-level), option-critic architecture.
- Modern: hierarchical planners in LLM agents.
Check yourself — multiple choice
- Same as flat
- High-level policy over sub-goals + low-level policy of skills; temporal abstraction + sample efficiency + skill transfer; Options / FeUdal / HIRO / option-critic; LLM agents modern extension
- Random
- Not real
HRL: high-level (sub-goals) + low-level (skills); temporal abstraction + transfer.
#policy-methods#planning
Practise Reinforcement Learning
214 interview questions in this topic.