Stop paying for the same tokens twice

I ran five reviewer agents over a 16K-token PR. The naive setup cost $1.32. Two architectural changes brought it to $0.49. Here's the bill, and what the experiment surprised me with along the way.

Five specialist agents reviewed the same pull request. Security, tests, perf, observability, API design. Each one re-read the full diff. Each one paid full price. The bill landed at $1.32 for a single PR pass on Claude Opus 4.6.

One PR. One review pass. Buck thirty-two.

If you're running multi-agent code review at any scale, you're bleeding money on the exact same tokens, over and over, paid in full each time.

I built a harness to measure the bleed and test the two obvious fixes. The numbers were sharper than I expected, and one of them inverted my assumptions about prompt caching.

What I built

A small Python harness with three modes and one shared 49K-character pull request diff. Five reviewer roles, each with a tight system prompt:

REVIEWER_ROLES = {

    "security":      "Auth, injection, secrets, access control...",

    "tests":         "Missing tests, skipped tests, weak assertions...",

    "perf":          "N+1 queries, sync I/O in request paths...",

    "observability": "Logging hygiene, error handling, missing metrics...",

    "api-design":    "Endpoint naming, status codes, idempotency...",

}

Three modes against the same diff:

1. 
   Naive. Each reviewer makes its own call with the full diff in the user message. Role text comes first so the prefix differs per call. Nothing gets cached.


2. 
   Cache. Each reviewer puts the diff in the system message, identical across all five calls. The first call writes to the provider's prompt cache. Calls two through five read from it at a 90% discount.


3. 
   Librarian. A first pass digests the diff into a compact JSON summary. Reviewers run against the digest, not the raw diff. Inspired by Cho et al.'s arXiv paper . Three Python files, one diff scenario, one pricing module. Run it with:
   
   
   
   
   
   
   

   CODE Kopieren

   export COPILOT_OAUTH_TOKEN=$(sqlite3 ~/.copilot/data.db \
   
     "SELECT access_token FROM github_accounts WHERE is_default=1;")
   
   
   
   python demo.py all --verbose --diff scenario/sample_pr_large.diff
   
   

   
   
   
   
   
   
   
   

   If you have the GitHub Copilot CLI installed, you already have an OAuth token sitting in ~/.copilot/data.db. Point the OpenAI SDK at https://api.githubcopilot.com with the Copilot-Integration-Id: copilot-cli header and you get the full model catalog (Anthropic, OpenAI, Google) on the chat-completions shape, with prompt_tokens_details.cached_tokens reporting. No PAT, no token exchange. About 30 lines of glue:
   
   

   
   
   
   
   

   CODE Kopieren

   import sqlite3, pathlib
   
   from openai import OpenAI
   
   
   
   con = sqlite3.connect(
   
       f"file:{pathlib.Path.home()}/.copilot/data.db?mode=ro", uri=True
   
   )
   
   token = con.execute(
   
       "SELECT access_token FROM github_accounts WHERE is_default=1"
   
   ).fetchone()[0]
   
   
   
   client = OpenAI(
   
       base_url="https://api.githubcopilot.com",
   
       api_key=token,
   
       default_headers={"Copilot-Integration-Id": "copilot-cli"},
   
   )
   
   

   
   
   
   
   
   
   
   

   One billing note: GitHub Copilot moves to usage-based billing on June 1, 2026. Under the new model, token consumption maps directly to GitHub AI Credits at Anthropic's published API rates, so the dollar figures in this article apply to Copilot users now too, not just people hitting the API directly. The architectural takeaway is the same either way. If you're on a paid Copilot plan, these savings are real dollars out of your monthly credit allotment.

   
   
   

   
   
   
   What I'd actually ship
   

   
   
   

   If I were rebuilding a review pipeline tomorrow with this data on the table:

   
   
   
   
   
   1. Move all shared context into the system message. It's the cheapest refactor in software, and it costs you nothing when the cache doesn't activate.
   
   
   2. Build the librarian pass once you cross three reviewers, or once your shared context is large enough that the activation threshold actually kicks in. The digest pays for itself across cold sessions.
   
   
   3. Stop treating cache hit rate as a constant. Measure it on your payloads, in your environment, against the model you ship with. Write a fail-loud guardrail and let it scream when assumptions break.
   
   
   
   
   

   Cho et al.'s paper measures GPU energy savings on SWE-Bench Verified, up to 25% on multi-agent runs. My harness measures wall-clock dollars on one PR review pass. Different proxy, same lesson: persistent shared context across agents is a free architectural lever sitting on the floor.

   
   
   
   
   
   

   Paper: Cho, Choi, Heo, Choi. of the repo.

   Vollständiger Original-Bericht

   Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

   ↗ Original-Artikel auf dev.to lesen