How does posterior sampling drive exploration in deep RL?
hardAnswer
- Maintain posterior over Q or model parameters.
- To choose action: sample θ ~ , act greedily w.r.t. sampled θ.
- Automatic exploration/exploitation via posterior uncertainty.
- Beta-Bernoulli for binary bandit → closed-form.
- Deep RL: Bootstrapped DQN samples one of K Q-heads per episode (approximate posterior); Bayesian NN with Gaussian variational posteriors; noisy nets.
- Regret-optimal in many settings.
Check yourself — multiple choice
- Random
- Sample θ ~ posterior, act greedy → automatic exploration via posterior uncertainty; Beta-Bernoulli for bandits; Bootstrapped DQN / noisy nets approximate for deep RL
- Same as UCB
- Not real
Thompson: sample from posterior, act greedy; automatic exploration.
#exploration#bayesian
Practise Reinforcement Learning
214 interview questions in this topic.