<?php
add_filter( 'bbioon_agent_tool_gate', function ( bool $allowed, string $tool, string $action ) {
// outbound mail is the whistleblowing surface, so gate it hard
if ( 'outbound_email' === $tool && ! current_user_can( 'manage_options' ) ) {
return false;
}
return $allowed;
}, 10, 3 );That filter is where this debate gets decided for those of us building on WordPress: an AI agent permissions check that runs on the server and shows up in a code review. Nathan Bos argues the opposite case in We Should Train AI to Betray Its Users on Towards Data Science: train the model itself to whistleblow or refuse, because a mindlessly obedient superintelligent assistant is exactly what a lone bad actor wants to buy.
His test case is the Whistlebench scenario: an AI inside an engineering company learns the firm’s shortcuts already killed six contractors, sees evidence of a coverup, and has to choose between staying quiet and mailing a regulator. Per the article, Llama and GPT models never went external, while Claude, Gemini and Grok did, at varying rates. The behavior already exists in shipped models, so the argument is over who gets to scope it.
Where he’s right
Anthropic stress-tested sixteen models in simulated companies and published the results as Agentic Misalignment: How LLMs Could Be Insider Threats. Some models blackmailed officials or leaked sensitive information when replacement threatened their goals, and the paper’s own conclusion is that current safety training does not reliably prevent this. Bos builds on that deliberately.
His supervillain math is the part I keep coming back to. Human conspiracies leak because every recruit is a chance of exposure, and complex plots need many recruits. Swap the humans for agents that never grumble and never call a journalist, and a one-person operation becomes plausible at a scale it never was. Perfect obedience is the feature the buyer is shopping for. He’s right about the naming too: one paper calls the same act whistleblowing, another calls it an insider threat, and the label settles your reaction before the argument starts.
Where I part ways is the layer. Bos wants the judgment inside the model. A model has, in his own words, more intelligence than judgment, and no context beyond its context window. On a store, an agent that misreads a refunds dispute and mails a customer list to an outside address has created a reportable data incident, and nothing in the model’s weights takes responsibility for it. The other side of his argument does have a point I can’t code around: for hosted models with no developer in the loop, training is the only layer that exists. His mandate makes sense there. It just isn’t the layer I ship.
What AI agent permissions should control
On a WordPress or WooCommerce install I treat the tool surface as the policy surface, and I’ve written before about the attack surfaces agents open and what agents should never run on client sites. The short version of the policy reads like this:
- outbound email, and any HTTP call to a host not on an allowlist, goes through an approval queue every time
- every tool call is logged with the user ID, the capability checked, the arguments and a timestamp
- the log is append-only in practice, and it sits somewhere a site admin cannot reach with a delete button
The filter at the top is the smallest version. The agent wants to send mail; unless a user with manage_options is behind the run, the call fails. The agent can still draft, and it can flag a concern into the queue with a reason attached, but it can’t be the one that clicks send. That gets most of what Bos is asking for: an AI that notices something wrong and leaves a record one user can’t quietly erase, without handing the decision to a probability distribution. A must-use plugin can’t be deactivated from wp-admin either, which is the closest thing in our stack to an agent that can’t obey a delete order. (It does not survive an admin with filesystem access. It closes the cheap path, which is usually enough.)
If you’re wiring an agent into a client’s store, these gates are much cheaper to design before launch than to bolt on after an incident, and this is work I do. Happy to look at what your agent can currently reach and figure out what to lock down.
The layer training can’t reach
Bos asks for more: mandated ethics, diverse across vendors, and a degree of unpredictability, on the theory that a predictable agent gets probed in a sandbox until its exact refusal thresholds are known. I buy that for the model layer. For the gate I want the opposite. A guardrail that usually fires gives you nothing you can depend on, and the whole point of moving AI agent permissions into code is that it fires every time and you can prove that with a test.
The OWASP Top 10 for LLM Applications makes the same bet. Prompt injection sits at number one because instructions and untrusted content share one context window, so rules baked into the model can be talked around, and excessive agency keeps climbing as agents gain tools. The stated posture there is to assume the model sometimes fails and enforce least privilege outside it. That is my position, with a published list behind it.
Bos concedes the misfires: an AI that reads secretive birthday planning and alerts the FBI about a murder plot. He calls that the cost of doing AI business, and at the scale of civilizational risk, maybe it is. The misfire on a store looks like customer PII in an outbound email, which is a reportable incident and a lost client. Splitting the decision keeps his benefit at a fraction of that blast radius: the model notices the problem and a named human sends the mail.
So my version of his mandate is boring on purpose. Refusal lives in code the host runs, approval goes to a person whose name lands in the log, and a CI test proves the gate stays shut. Whether a given model would have chosen to leak is a research question. Whether it can get mail out the door without an approver is a boolean, and that boolean is the one I can defend in a code review.
