POMDP — how does it differ from MDP?
hardAnswer
- Partially Observable MDP: agent observes which gives incomplete info about state .
- Add observation function .
- Belief state b(s) = posterior over s given history → sufficient statistic for optimal policy.
- Solving POMDPs is PSPACE-hard exactly; approximate via RNN/LSTM encoding of history, or particle filter beliefs, or transformer over recent observations.
- Realistic robotics / partial-info games.
Check yourself — multiple choice
- Same as MDP
- Observation o gives partial info about hidden state s; belief b(s) = posterior sufficient statistic; approximate with RNN / particle filter / transformer
- Random
- Not real
POMDP: partial observations + belief state; approximate via RNN / transformer.
#fundamentals#theory
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- What is the Markov property and why does it matter?
- How do you handle partial observability in deep RL?
- How do you choose the discount factor, and what goes wrong at the extremes?
- Define the RL setting: agent, environment, state, action, reward, policy.
- On-policy vs off-policy learning — what's the difference?
- What does the Bellman equation say?