You want to optimize which of five banners to show. Bandit or full RL?
mediumAnswer
- A contextual bandit, because showing a banner does not meaningfully change the state the next user arrives in, so there is nothing to credit across time.
- That single simplification removes the hardest part of reinforcement learning and gives you a problem with clean theory, fast convergence, and well-understood algorithms such as Thompson sampling or upper confidence bounds.
- It also fails gracefully, since the worst case is exploring a mediocre banner slightly too often.
- Full reinforcement learning becomes appropriate if the decision has genuine downstream consequences on the same user, such as sequencing a multi-step onboarding flow, where an early action changes what later actions are available or effective.
Check yourself — multiple choice
- Full RL, it is more powerful
- A contextual bandit: the action does not change future state, so no temporal credit assignment is needed and Thompson sampling converges fast and fails gracefully
- Neither, use A/B testing only
- Bandits cannot use context
No state transition means no need for sequential credit assignment, so a bandit is the right and much simpler tool.
#applications#exploration
Practise Reinforcement Learning
214 interview questions in this topic.