When does tabular Q-learning fail in practice?
mediumAnswer
- (1) State space too large: even discretized MazeGrid > 1M states makes tabular infeasible.
- (2) Continuous features: discretization fights curse of dimensionality.
- (3) Function structure across states unexploited (Q for nearby states should be similar → NN generalizes; tabular treats them independently).
- (4) Slow: no generalization means each state must be visited separately.
- Every real RL problem > toy uses function approximation (usually NN).
Check yourself — multiple choice
- Never fails
- Fails when state space large / continuous / structured (NN can share params, tabular can't) — real RL uses function approximation
- Random
- Only supervised
Tabular fails: large / continuous / structured states — needs FA.
#theory#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.