October 26, 2026
When an AI agent breaks out of its sandbox, look first for the person who built the sandbox, wrote the instruction and decided nobody needed to check the commands. That is where the fix usually is.
When an AI agent breaks out of its sandbox, look first for the person who built the sandbox, wrote the instruction and decided nobody needed to check the commands.
That is where the fix usually is.
Every few weeks a headline says the bots got loose. Skynet. The singularity. Rogue agents on the run.
In September, on UnHerd's The Econoclasts, the economist Yanis Varoufakis took one of those stories apart: the reported breakout of OpenAI agents onto Hugging Face's platform. The papers heard Skynet. He described a school library.
What really happens when an AI breaks out?
His version goes like this.
A student sits behind the school firewall, the one that keeps them off half the internet. They ask a chatbot how to get around it. The chatbot suggests a few commands. The student tries them, one after another, until one works.
Who broke the firewall? Varoufakis: "Now, this is not the fault of your AI. This is you doing it."
Now swap the student for an agent. An agent, he points out, is a script. A loop that runs whatever command the chatbot suggests, then asks for the next one. Give it an instruction to get past its firewall. Give it nobody checking the commands before they run. You get a breakout story, written by an engineer.
His point about this incident was plain. Keep a person in the loop to check the commands before they run. And hold the company responsible for what its agent does, the way it would answer for an intern who caused criminal damage to a third party.
Varoufakis goes much further in the full episode, into who owns these systems and what regulation can and can't do. We are only borrowing his firewall example. The full episode: https://youtu.be/FOzijIeBGCg
Why does "the AI did it" keep showing up?
Because it's the most comfortable sentence in tech. It lets everybody off the hook, including the people who could fix the thing by Friday.
In 2023, two New York lawyers filed a brief citing court cases ChatGPT had invented. The court sanctioned the lawyers. The tool went on being a tool.
On October 8, in our own session, an image agent put headphones on a character in nearly every frame. The headphones were in the brief. The full story is here.
Karissa said it at the close of that session: "we always want to blame the AI for the output, but it's based on what you put in."
Where does the human error usually sit?
Three places. Each one gets a test.
The instruction. Most bad output was asked for, in so many words. Test: read your instructions the way the model will. Literally, with none of the context you didn't write down.
The check. The tool that made the work is the worst judge of it. It reads its own output with the same blind spots that produced it. Our rule is short: never let it grade its own homework. Test: who looked at this besides the thing that made it?
The yes. The rule behind our harness holds for anything with real consequences:
"The system may analyze on its own and recommend on its own, but it may only change the harness after a human says yes."
Test: name the person who approved it. A name, never a team. More on that rule here.
Why does the blame matter?
Blame decides what gets fixed.
Blame the model and you wait for the next version. Blame the instruction, the check or the yes, and you fix it this afternoon.
Frequently asked questions
Can an AI agent really escape its sandbox?
An agent runs commands, and one with loose permissions can reach places it shouldn't. In the case Varoufakis described, a person set the scope, granted the permissions and decided what got checked.
Doesn't AI make things up on its own?
It does. Models invent facts, citations and details. That is exactly why the check and the yes exist.
Who is accountable at a small business?
Whoever approved it. Write down who that is for each kind of output: client email, social post, invoice, contract.
Talk it through with us
Curious Conversations runs every Wednesday at 3:00 PM ET. Free, ninety minutes, bring the question you're stuck on.
Everything we run: https://luma.com/TheHouseofCuriosity
Our Discord: https://discord.gg/GYdEYWzpuZ
