How do you handle partial observability in deep RL?
mediumAnswer
- (1) State-stacking: last k observations as input (DQN frame stack).
- (2) RNN / LSTM policy: hidden state summarizes history (R2D2).
- (3) Transformer over recent history (Decision Transformer).
- (4) Belief-state approximation via VAE or particle filter.
- (5) Reward-conditioned models.
- Rule: for weakly non-Markovian (short history matters), stacking + short RNN suffices.
- For long-horizon dependencies, transformer or explicit memory.
Check yourself — multiple choice
- Ignore
- Frame stacking / RNN-LSTM / transformer over history / VAE belief / particle filter; short history = stacking, long-horizon = transformer / explicit memory
- Random
- Not possible
POMDP handling: stack / RNN / transformer / belief; scale by history length.
#fundamentals#theory
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- What is the Markov property and why does it matter?
- POMDP — how does it differ from MDP?
- How do you choose the discount factor, and what goes wrong at the extremes?
- Define the RL setting: agent, environment, state, action, reward, policy.
- On-policy vs off-policy learning — what's the difference?
- What does the Bellman equation say?