What role does the discount factor gamma play?
easyAnswer
- Gamma in [0, 1) weights how much future rewards matter compared to immediate ones.
- Gamma close to 0 = myopic (only care about immediate reward); gamma close to 1 = far-sighted (care about long-term).
- Also ensures the return sum converges in infinite-horizon problems.
- Choose based on the task horizon and stability of value estimates.
Check yourself — multiple choice
- always leads to divergence in episodic tasks
- makes the agent myopic
- Gamma controls how much a policy explores
- Gamma only matters in policy-gradient methods
makes the agent care only about the current reward.
#fundamentals
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- Define the RL setting: agent, environment, state, action, reward, policy.
- On-policy vs off-policy learning — what's the difference?
- Model-based vs model-free RL — when do you prefer each?
- What is the Markov property and why does it matter?
- POMDP — how does it differ from MDP?
- V(s) vs Q(s, a) — the difference.