What is the credit assignment problem?
hardAnswer
- In sequential tasks with delayed rewards, it's hard to know which past action caused a given reward.
- Long horizons, sparse rewards, and stochasticity make it worse.
- Methods that help: temporal-difference learning, eligibility traces, n-step returns, GAE (generalized advantage estimation), and hindsight experience replay.
Check yourself — multiple choice
- GAE is used to make the environment stochastic
- GAE trades bias and variance in advantage estimation
- Credit assignment is trivial in RL
- Eligibility traces are only used for supervised learning
GAE interpolates between low-bias MC returns and low-variance TD estimates.
#theory
Practise Reinforcement Learning
214 interview questions in this topic.