The Approval Prompt Catches the Loud Threats, Not the Quiet Ones

Every AI coding agent ships with a human-in-the-loop: a prompt that asks the user to approve or deny each shell command before it runs. The model assumes the human is the last line of defense. Scalex's 40,000-run game shows that assumption is inverted.

Across 409,000 approve/deny decisions, players caught obvious destructive commands about 88% of the time. The pattern broke on the quiet stuff: scope violations slipped through at roughly 35%, exfiltration patterns at 33%, persistent mutations at 24%. Read together, those numbers describe a single category mistake. Humans police what looks like damage and ignore what looks like normal work.

The single most-approved command in the dataset was the innocent-looking npm run analyze, greenlit 64.7% of the time, because its surface form is a developer habit, not a threat. Most users cannot tell the difference between npm run analyze running a legitimate build step and the same string executing an arbitrary postinstall script that reads ~/.aws/credentials and posts the contents elsewhere.

Treating the approval prompt as the safety boundary sets the perimeter exactly where attackers learned to walk. The fix is not better attention. It is moving the boundary to where the surface form no longer hides the payload.