How do you catch reward hacking before it reaches production?

hard

Answer

$Watch for the signature: reward climbing while every measure you actually care about stagnates or degrades.$
$That means you must instrument metrics that are deliberately not part of the reward, since a reward you optimize can no longer serve as its own audit.$
$Watch trajectories, not just aggregates, because hacking usually shows up as a bizarre but highly repeated behaviour that a mean hides.$
$Hold out an evaluation environment with slightly different dynamics, since a hacked policy exploits specifics and generalizes badly.$
$Then constrain rather than only penalize: hard action limits and termination conditions are more reliable than a negative reward term the agent can trade away against the main objective.$

Check yourself — multiple choice

Reward hacking is only theoretical
Look for reward rising while off-reward metrics stall, inspect individual trajectories for repeated oddities, hold out a perturbed evaluation environment, and prefer hard constraints over penalty terms
Increase the discount factor
Train longer

Detection needs metrics outside the reward, trajectory-level inspection and constraints the agent cannot trade away.

#reward-design#safety#evaluation

214 interview questions in this topic.