How is a recommender system a bandit / RL problem?
mediumAnswer
- Contextual bandit: user context = state, item shown = action, click/watch = reward.
- Long-term reward → full RL for session-level or LTV optimization.
- Challenges: (1) partial feedback (only chosen item observed).
- (2) Non-stationarity (user preferences drift).
- (3) Delayed rewards (retention).
- (4) Off-policy evaluation critical (can't A/B every candidate).
- Modern: Thompson sampling for exploration, contextual bandits for retrieval, RL for long-horizon (Netflix, YouTube, TikTok).
Check yourself — multiple choice
- Random
- Contextual bandit: user context = state, item = action, click = reward; full RL for long-horizon LTV; challenges: partial feedback + drift + delayed + OPE; Thompson / RL at Netflix / YouTube
- Same as CF
- Not real
Recsys as bandit/RL: user=state, item=action, click=reward; Thompson + off-policy eval.
#applications
Practise Reinforcement Learning
214 interview questions in this topic.