I Built an AI Agent That Breaks Your API and Fixes It Automatically

This is the third post in a series. The first covered

why APIs break in production. The second covered

the plan for building PatchFlow. This one covers

what actually happened.

The honest version of how this got built

I started this hackathon with a clear plan and spent the first three days

completely rebuilding it.

The original idea was a cloud operations monitoring system. Agents watching

infrastructure, detecting anomalies, remediating incidents. It was a solid idea

with one problem: Alibaba Cloud had just shipped their own native operations

monitoring product. I found out on day three.

So I scrapped it and started thinking about what I actually found annoying

as a developer.

The answer came quickly. Every production incident I had ever dealt with traced

back to the same thing: the API did not handle failure gracefully. Not because

the developer was careless, but because testing failure cases manually is slow,

repetitive, and easy to skip when you are trying to ship.

I had written about this exact problem in my first post. The irony of ignoring

my own premise was not lost on me.

That became PatchFlow.

What PatchFlow does

You give it your API. It breaks it in 18 different ways, finds every endpoint

that does not handle failure gracefully, reads your actual source code, and

opens a GitHub pull request with the fix already written.

The full pipeline runs through six agents:

1. 
   Discovery parses your OpenAPI spec or Postman collection into a grouped
   endpoint list for you to review before anything runs


2. 
   Chaos injects real failure modes via actual HTTP requests, not mocks


3. 
   Analyst identifies patterns across all results and produces a risk score


4. 
   Repo Context clones your repository and follows the call chain across
   files to find where the real problem lives


5. 
   Fix writes a patch using your actual code as context, not a generic template


6. 
   Review audits the fix before any PR is opened, rejects it if something
   is wrong, and sends it back to Fix with specific feedback

That last pair, Fix and Review, ended up being the most important thing I built.

The part that actually took the longest

The Repo Context agent.

The naive version of this problem sounds simple: find the file that handles

the failing endpoint, read it, write a fix. That works for about thirty percent

of real codebases.

The other seventy percent have route handlers that look like this:

@app.post("/payments/charge")

async def charge(body: ChargeRequest, db: Session = Depends(get_db)):

    return await payment_service.process(body, db)

The route itself has nothing wrong with it. The problem is three files away,

inside payment_service.process, which calls stripe_client.charge, which

has no timeout configured and no exception handling around the external API call.

Teaching an agent to follow that chain required building a set of tools the

agent could call iteratively: search_in_files, read_source_file, and a

reasoning loop that decided at each step whether it had found the actual

problem location or just a delegation point.

The Qwen ReAct loop was what made this possible. The agent would read a file,

reason about whether the bug lived there or further down, decide to follow an

import, read the next file, and repeat until it reached a conclusion. Each step

was a genuine reasoning decision, not a heuristic.

I used qwen3.7-plus via Qwen Cloud's OpenAI-compatible endpoint for every

agent in the pipeline. The tool-calling support was solid throughout. I did not

have to change any of the tool schemas between agents, which kept the shared

base agent class clean.

The Fix and Review loop

After Repo Context locates the actual code, Fix writes the patch. But the first

version of a generated fix is rarely the best version.

The Review agent runs independently and checks:

• Does the fix address the actual failure mode that was found


• Are all required imports present and correctly referenced


• Does the fix introduce duplicates that already exist in the file


• Does the style match the conventions of the surrounding codebase


• Could this change break existing behaviour

If Review finds a problem, it rejects the fix with structured feedback and

sends it back to Fix. Fix revises. The loop continues until Review approves.

This mirrors what real code review actually does. The difference is that the

entire cycle happens before the pull request is opened. By the time a PR lands

in the developer's queue, it has already been written and reviewed by two

separate agents with different objectives.

In testing against my demo application, the rejection rate was around thirty

percent on the first attempt. Most rejections were for missing imports or

duplicate exception handlers. After one revision cycle, the approval rate was

close to a hundred percent.

The part I got wrong

Endpoint discovery.

I spent two days on approaches that did not work. Scanning the codebase for

route decorators worked reasonably well for FastAPI but broke on Express, missed

dynamically registered routes, and consumed a lot of tokens on large codebases.

Probing common URL patterns was worse.

The solution was embarrassingly obvious once I stopped overthinking it: use the

same inputs that Postman and Insomnia use. OpenAPI specs, Postman collections,

or manual entry. Every serious API team already has one of these. Stop guessing

and ask for what you need.

This also led to one of the better UX decisions in the project. Instead of

auto-selecting endpoints and running immediately, PatchFlow shows you every

endpoint grouped by tag, flags anything that looks risky (admin routes, delete

operations, webhooks) as unchecked by default, and waits for you to confirm

what to test. Nothing runs until you say so.

I borrowed this from Postman because it is the correct mental model: you see

the full inventory before executing anything.

What Qwen Cloud made possible

I want to be specific about this rather than generic.

The ReAct loop with tool calling is where Qwen's quality showed up most clearly.

Multi-step reasoning tasks where the agent had to gather information across

multiple tool calls, form a hypothesis, test it with another tool call, and then

reach a conclusion were handled well. The Repo Context agent in particular runs

four to six tool calls per finding before concluding, and the reasoning between

calls was coherent.

The OpenAI-compatible endpoint made it straightforward to build a shared base

agent class that all six agents inherit from. The tool schema format is identical

to what you would write for GPT-4, which meant I did not have to learn a new

interface while also building the product logic.

The one adjustment I made was being more explicit in system prompts about the

expected output format. Asking for JSON output with a specific schema and

including a concrete example in the prompt produced more consistent structured

responses than relying on implicit formatting expectations.

The numbers

• 18 failure modes across four categories (network, dependency, data, resource)


• 6 agents in the pipeline


• 6 database tables (sessions, endpoints, failure results, agent steps,
  reports, pull requests)


• Every agent step streamed to the frontend in real time via WebSocket

Against my demo application, a typical run across six endpoints produces around

forty failure injection results, identifies three to five findings, and generates

two to four pull requests. The full pipeline takes four to six minutes to complete.

Where it stands

PatchFlow is live at the link below. You can point it at any API using an

OpenAPI spec URL, upload a spec file, or use a Postman collection.

The GitHub integration uses your own OAuth token, so every pull request it opens

appears under your account, not a service account. You review and merge from the

PatchFlow dashboard.

The three things I would build next if I had more time: scheduled scans triggered

on every deployment, broader language support beyond Python (the Fix and Review

agents currently work best on FastAPI codebases), and Review agent metrics

tracking which failure modes produce the highest fix rejection rates.

Closing thought

The most useful thing I learned building this was not technical. It was that

rejection is a feature.

The Fix agent's first output is almost never its best output. The Review agent

exists because first drafts have problems, and structured feedback improves them.

That principle applies equally to agent pipelines and to the code they are

fixing.

If you are building a multi-agent system and your agents only produce outputs,

consider adding one whose entire job is to push back.

Built for the Global AI Hackathon Series with Qwen Cloud.

Source code: github.com/jaytech504/chaos-agent

Live demo: PatchFlow