Safe RL — how do you constrain during learning?
hardAnswer
- (1) CMDP (Constrained MDP): maximize reward subject to ≤ threshold.
- Solved via Lagrangian or projected gradient.
- (2) Safe exploration: pretrain with safe demonstrations, use safety layers.
- (3) Reachability analysis.
- (4) Reward shaping with constraint violations.
- (5) Test-time verification: reject actions predicted to violate constraints.
- Standard in robotics, autonomous vehicles, medical RL — high-stakes safety-critical domains.
Check yourself — multiple choice
- Ignore safety
- CMDP (constrained MDP) via Lagrangian, safe exploration + safety layers, reachability analysis, constraint reward shaping, test-time verification — high-stakes domains
- Random
- Not real
Safe RL: CMDP + Lagrangian + safe explore + reachability + verification.
#safety
Practise Reinforcement Learning
214 interview questions in this topic.