What actually breaks when you take an AI coding agent from demos to production — and what held up.
How I got here
I'm not an engineer. In January I was a business owner who couldn't write a line of code and didn't know what a database schema was. Six months later, the complete software stack of 12 small companies — multi-tenant ERPs handling live purchase orders and inventory, customer-facing SaaS products, PWAs running on shop-floor tablets, e-commerce sites — runs on software built and operated almost entirely through Claude Code.
This is not a "look how easy AI makes everything" post. The agent writes better code than I ever will. What it doesn't do — what nothing does for you — is build the habit of verification. Every painful incident I've had traces back to a check that nobody (me) thought to run. The agent is rarely the bottleneck; the absence of an audit habit is.
Here are the three lessons that cost me real production incidents, and the boring, repeatable procedures that now prevent them.
Lesson 1: Staging environments lie the moment you stop enforcing them
Early on I did what the tutorials say: spin up a staging clone, test there, then deploy. For a while it worked.
Then a release passed staging clean and broke production anyway. Root cause: weeks earlier, a manual fix had been run directly on staging and never back-ported. The schema had quietly diverged three migrations back. The clone wasn't a clone anymore — it was a diorama.
The fix wasn't "maintain staging harder." Parity is a discipline, not an environment, and a solo operator will always lose that discipline race. What actually holds up:
- Feature flags so risky changes ship dark and enable separately from deploying
- A pre-flight check that fails loudly if migration state doesn't match expectations before anything runs
- A fast, rehearsed rollback path — not a theoretical one
- Testing against production under guardrails: tagged test emails, zero-amount or sandboxed payments, and a baseline snapshot so every piece of test data is identifiable and removable afterward
A "safe" environment nobody audits is worse than no safe environment, because it manufactures false confidence.
Lesson 2: Silent mutation failures are the #1 trust killer
The incident that burned the most goodwill wasn't a crash. It was a save button.
It returned success. The UI looked fine. But the write had failed a validation check three layers down, and the error was swallowed — caught, logged nowhere useful, surfaced to no one. The user assumed their data was saved. They found out a week later when a number on an invoice was wrong.
Here's the thing about non-technical users (I was one, six months ago): they cannot distinguish "broken" from "slow" from "fine" without an explicit signal. A quiet data loss with no error banner doesn't read as a bug — it reads as the new system can't be trusted, and that reputation never fully recovers.
The rule that came out of it: every write path surfaces success or failure to the user, no exceptions. No silent 200s on failed writes. No bare catch {}. During testing, loud and annoying beats quietly wrong, every single time.
The audit for this is mechanical enough that an agent does it well: inventory every mutation entry point (server actions, API handlers, RPC calls, form submits), classify each one as surfaced or silent, fix the silent ones with one consistent error-to-UI convention — then verify by forcing failures (kill the network, revoke a permission, send bad input) and confirming red state actually appears.
Lesson 3: Permission bugs live in the corners nobody clicks
None of my access-control bugs were in main flows. Main flows get exercised hundreds of times a day; they self-correct fast.
The bugs were in the corners: the settings page nobody visits, the admin route that got a new sibling page months later which never inherited the guard, the API endpoint a button used to call — the button is gone, the endpoint is still live. One of those had been quietly exploitable for months.
The only thing that catches these is a full route × role sweep, done systematically: enumerate every route (crawl the running app — static analysis misses dynamically-registered pages), enumerate every role, drive a real logged-in session through every combination, and attempt the forbidden writes directly — not just checking whether a button is hidden, but whether the server actually refuses the request. Then persist the harness in the repo so the next sweep is one command instead of an archaeology project.
"I think I tested that" is not a security model.
What made this scale to 12 companies instead of collapsing at 3
- Checklists beat cleverness. The same five or six audit types, applied identically everywhere. Nothing bespoke.
- The agent's output is a draft until verified. Every time. Not just when something feels risky — feelings are exactly what miss the corner cases.
- A human stays in the loop for the irreversible: production data mutations, money, anything touching customer PII.
If you're starting this today
Start the audit habit before the first incident, not after. Small, boring, repeatable checks — schema drift, silent errors, route permissions, migration safety — beat heroic one-off code reviews every time.
I eventually packaged the exact workflows I run into 12 Claude Code skills (drop-in .claude/skills/ folders). The deploy-failure-diagnosis one is free — pay what you want, $0 is fine if you want to see the format. The full pack is $29. One line of pitch and no more: it's the audits I actually run, not a course.
Disclosure
This article was drafted with Claude's help and edited by me. The incidents are real and happened on real systems; details are genericized to protect the businesses involved. If you have questions about any specific failure mode, ask in the comments — that's the part I actually enjoy talking about.
SOCIAL SHARE CARD GENERATOR