Building Diagram AI: From Natural Language (and Screenshots) to AWS Architecture Diagrams

Drawing architecture diagrams by hand is tedious. You drag boxes around, align icons, fix arrows, and then your design changes and you do it all again. I wanted something where I could just describe an architecture in plain English — or paste an existing diagram — and get a clean, AWS-style diagram back.

This post is the story of building Diagram AI: a browser app that turns natural language and images into architecture diagrams. It runs on Vercel, uses Amazon Bedrock (Claude Haiku 4.5) for the language understanding, and renders the actual diagrams with AWS's open-source

The core idea

The pipeline is simple to describe:

[User input: text or a diagram screenshot]

        │

        ▼

[Vercel · Next.js]

   ├─ /api/generate-yaml  → Bedrock Claude Haiku 4.5 → diagram-as-code YAML

   └─ /api/render         → AWS Lambda (Function URL) → awsdac → PNG

        │

        ▼

[Browser: live preview + PNG download]

The clever part is the middle layer. Instead of asking an LLM to "draw a diagram" (which it can't do reliably), I ask it to write YAML in the diagram-as-code schema. A deterministic CLI then renders that YAML into a real, standards-compliant AWS diagram. The LLM does the understanding; a proven tool does the drawing.

Why diagram-as-code?

diagram-as-code (the awsdac CLI) is an AWS Labs project that generates architecture diagrams from human-readable YAML. It ships official AWS service icons and follows AWS diagram conventions. Crucially, it's extensible: you can add your own definition files to draw non-AWS services too. That extensibility became important later.

First wall: you can't just import it

My initial plan was elegant on paper: diagram-as-code is written in Go, so I'd compile it into a Vercel Go Serverless Function and call it as a library. Clean, all-in-one, no extra infrastructure.

Then I read the source. Almost all of the core logic lives under Go's internal/ directory — internal/ctl, internal/types, internal/definition, and so on. Go's language rules forbid importing internal/ packages from outside the module. The library-import approach was simply impossible.

So I pivoted: run awsdac as a CLI inside AWS Lambda, exposed via a Function URL, and have the Vercel app call it over HTTPS. Vercel handles the frontend and the LLM; Lambda handles the rendering.

Second wall: no Docker, no Go on the machine

The plan was to package the Lambda as a container image. But the build machine had neither Docker nor Go installed. Rather than fight that, I switched to a zip-based Lambda on the Python 3.12 runtime:

• Download the official prebuilt awsdac Linux binary from the GitHub releases.


• Write a tiny Python handler that shells out to the binary.


• Zip the two together and upload. No Docker, no Go build step.

The handler is small — receive YAML, write it to /tmp, run awsdac, return the PNG as base64:

proc = subprocess.run(

    [AWSDAC_PATH, "--allow-untrusted-definitions", in_path, "-o", out_path],

    capture_output=True, text=True, env={**os.environ, "HOME": "/tmp"}, timeout=25,

)

Third wall: a 403 that wasn't in any policy

With the function deployed, direct aws lambda invoke worked perfectly — a clean PNG came back. But calling the Function URL over HTTPS returned 403 Forbidden / AccessDeniedException.

The auth type was NONE. The resource policy granted lambda:InvokeFunctionUrl to *. Everything looked right. I checked for org-level SCPs and RCPs — none.

The answer was in a documentation note: since October 2025, public function URLs require two permissions — both lambda:InvokeFunctionUrl and lambda:InvokeFunction. I only had the first. Adding the second statement fixed it instantly:

aws lambda add-permission --function-name diagram-ai-render \

  --statement-id FunctionURLAllowInvoke \

  --action lambda:InvokeFunction --principal "*" \

  --invoked-via-function-url --region us-east-1

Lesson: when a 403 makes no sense, check whether the platform's rules changed recently, not just your own config.

Making the LLM output good diagrams

Getting valid YAML was the easy half. Getting good-looking diagrams took several rounds of prompt engineering, each driven by an ugly output:

1. Arrows piercing containers. The LLM would draw an arrow from User straight to an ALB deep inside a VPC, slicing through the "AWS Cloud" and "VPC" boxes. Fix: a hard rule that any external→internal arrow must route through an Internet Gateway placed on the VPC's border (BorderChildren), exactly like AWS's own reference diagrams.




2. Arrows overlapping label text. diagram-as-code pins each label directly under its icon and routes arrows through the icon's center, so vertical flows always crossed the text. After testing four layout strategies, the winner was horizontal flow (Direction: horizontal): horizontal arrows pass through the icon's mid-height and clear the labels below.




3. Invented resource types. Asked for a "box," the model confidently produced AWS::Diagram::Container — a type that doesn't exist. I added an explicit list of safe AWS types plus a rule: never invent types; fall back to a generic resource if unsure.

Each fix went straight into the system prompt with a concrete good/bad example. The prompt grew long, but the outputs got dramatically cleaner.

Going beyond AWS

diagram-as-code is AWS-first, but its definition-file mechanism lets you add arbitrary icons. I wanted hybrid diagrams — "GitHub → Vercel → AWS Lambda" — to look right.

I built a small pipeline: pull SVGs from

What I'd tell my past self

• 
  Read the source before committing to an integration strategy. The internal/ package issue would have saved a day if I'd caught it on day one.


• 
  When auth fails inexplicably, suspect recent platform changes. The dual-permission Lambda URL requirement was invisible in my own config.


• 
  For LLM-generated structured output, encode taste as rules with examples. "Don't pierce labels" only worked once it became a concrete layout rule with a good/bad sample.


• 
  Same-origin hosting beats clever CDN tricks when you control the deploy anyway.

The stack, in one line

Next.js on Vercel · Amazon Bedrock (Claude Haiku 4.5, text + vision) · AWS Lambda (zip, Python 3.12) running the awsdac CLI · simple-icons + sharp for non-AWS icons · GitHub Actions for CI/CD.