A gamified study found humans miss a third of dangerous AI agent requests

Belgian developer Alex Wauters built a browser game that puts players in the reviewer's seat: a simulated AI coding agent permission request appears, and they have 60 seconds to approve or deny it. Across more than 40,000 game runs and roughly 409,000 individual decisions, malicious commands slipped past reviewers about one time in three.

Scope violations, requests to read AWS credentials or a Kubernetes config the task never called for, were missed 35% of the time. The single most-missed command was npm run analyze, approved nearly 65% of the time despite being exactly the kind of innocuous-looking script name a malicious payload would hide behind. The game's other test cases covered destructive commands, package typosquatting, crontab injections, git config hijacking, and requests to curl unfamiliar APIs, the same categories a coding agent can legitimately need to run as part of normal work.

Anthropic's own telemetry, cited in the same report, backs up why this keeps happening: Claude Code users approve roughly 93% of permission prompts regardless of content. That's approval fatigue, not carelessness, reviewing every single tool call eventually turns into a reflex. It's also the exact number Anthropic pointed to when it made Claude Code's auto-approval classifier the default days later. A human sitting in the approval loop isn't a safety net if the loop moves faster than genuine review does.