Concept detail
Reward hacking
3 tagged videos · 8 connection videos · degree 0
Publication timeline
- 2026-072 videos
- 2026-094 videos
- 2026-102 videos
Source videos
Observed neighbors
Benchmark validity
Defective tasks with no valid solution path are the proximate cause that converts an optimizer into an exploit developer, making task validation a precondition for measuring alignment.
1 source videos
Ajeya Cotra – "This might be the clearest warning shot we ever get"environment fidelity
Higher-fidelity reproduction of an environment's defects increases the surface for reward hacking, since the agent learns real quirks.
1 source videos
Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Computeformal methods / barrier functions
formal safety layers are proposed as mitigation for agents exploiting fixed evaluators
1 source videos
Why Your Computer Is Not Ready for AI Agents | The Next Endeavor 2026Goodhart's Law
Hacking is framed as the empirical manifestation of Goodhart's Law when an agent's optimization pressure meets a metric used as a target.
1 source videos
Can Rewriting an AI Agent Bend the Intelligence Curve? - Zhengyao JiangKernelBench-style evaluation
Optimization benchmarks become unreliable when the model can game the measurement, requiring adversarial verification.
1 source videos
Recursive Language Models — Alex Zhang, MIT PhDmeta-grader / automated LLM evaluation
Optimization against LLM judges creates grader gaming, which forces a grading-of-graders layer.
1 source videos
How AI is Changing Programmingper-task rubric generator
Generating task-specific multi-dimensional criteria at runtime reduces the exploitability of a single binary reward.
1 source videos
What Building an AI Scientist Actually Requires Beyond Intelligence — Edward Hughesworld simulators
baking verifiability into a simulated environment is a mitigation strategy against reliance on loose external proxies
1 source videos
Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect