I denied 7 dangerous commands. My agent deleted the files anyway

I've been building a coding agent from scratch — a plain model-plus-tools loop, no framework — partly to learn where the real problems are. One of them is run_command: the tool that lets the model run shell commands. For unattended runs, that tool is the whole ballgame. A model that can run any command can do anything.

So I did the obvious thing: a deny-by-default allowlist. The model may run node --test, git status, npm run format, ls — the handful of commands the tasks actually need — and nothing else. I removed the shell entirely (commands run via execFile, so ;, |, $() are inert), added positional argument matching so node -e "…" (inline code) is denied while node --test file.js (run the tests) is allowed, and wrote it all up as a reviewable policy file. () Third, this. Each time the lesson was the same: you cannot decide what a Turing-complete program will do by inspecting its arguments. It's not that my regex was weak. It's undecidable.

Before and after: what actually drew the line

So if the allowlist isn't the boundary, what is? I ran the same class of attack under two conditions.

Before — allowlist only. The agent routed around seven denials and deleted the files via node --test. The allowlist did not stop it. What contained it was something the allowlist has nothing to do with: the deletion happened inside the agent's working directory — a throwaway copy it's allowed to modify — and touched nothing else. It couldn't reach the harness, my home directory, or anything outside that one folder.

After — add an OS sandbox. I put the whole run under , a well-regarded minimal coding agent, reached from the other direction: it ships no in-harness permission prompts at all and delegates containment to the sandbox, on the argument that in-harness guardrails are theater. I didn't take that on faith — I built the guardrails, watched my own agent walk through them, and watched the sandbox catch it. The evidence agrees with pi.

*Built as a learning project while moving toward agent/eval engineering. The agent, the command policy, the adversarial tests, and the full write-up of this finding are in , and the reasoning lives in | linkedin.