I turned on prompt caching for an agent I run all day, watched the next invoice, and the number went up. Not by a rounding error — meaningfully up. This is the write-up of why, because the thing I got wrong is the thing almost every "just add caching" post skips.
The one-sentence version of caching
You send the same long system prompt (plus tool schemas, plus a pasted document) on every call. Without caching you pay full input price to make the model re-read that identical block every single time. Caching stores a prefix of your prompt and, on the next call, charges a fraction to read it back instead of full price.
Sounds free. It is not free. That's the trap.
The part that bit me: writes cost more
A cache entry has two prices, and I only knew about one of them:
Cache read (a hit): billed at as low as 0.1x the input rate. This is the number everyone quotes. ~90% off the cached tokens. Great.
Cache write: billed at a premium — around 1.25x the input rate on the short (5-minute) window. This is the number nobody mentions.
So the first time you send a prefix — or any time it's expired — you don't save 90%, you pay a 25% surcharge to store it. Caching only comes out ahead once that same prefix is read back, not just written. The good news: the write costs ~0.25x more than a normal read (1.25x vs 1x), and a single cache hit saves 0.9x (0.1x vs 1x) — so the very first reuse already earns the premium back, and every hit after that is ~90% off. The catch is you have to actually get the hits.
Which means the failure mode is specific and nasty: write a cache entry, then never read it. You paid extra for storage you never used. Do that on every single call and you've built a machine that pays a surcharge to cache things it immediately throws away.
That was exactly my bug.
What I actually did wrong
My "stable" system prefix wasn't stable. Buried near the top of it was this, which I'd added months earlier and forgotten:
Current session started: 2026-07-14T09:41:07Z
A timestamp. Regenerated every call. Which meant the prefix was different every call, so the provider dutifully wrote a fresh cache entry every single time — full write premium — and never once got to read one back, because no two prefixes matched. I had turned on the expensive half of caching and none of the cheap half.
The tell was right there in the response the whole time, I just wasn't looking:
print(resp.usage)
# cache_creation_input_tokens: 3948 <- writing every call
# cache_read_input_tokens: 0 <- never reading. there's your bug.
cache_creation climbing while cache_read sits at zero is the signature of a busted prefix. If you take one thing from this post: log those two fields and watch them.
The fix is discipline, not more caching
Caching rewards a prefix that is byte-for-byte identical across calls and sits before anything that changes. Two rules:
1. Static content first, dynamic content last. System prompt, tool definitions, the big pasted document — all up front and unchanging. The user's actual turn, the timestamp, the per-request ID — all at the end, after the cached region. Anything that varies must live downstream of the cache breakpoint, never inside it.
2. Keep the prefix stone-cold identical. No timestamps, no reordered JSON keys, no "helpful" per-request injection into the system block. One moving character and the whole prefix misses.
For Claude, you mark where the stable prefix ends with a breakpoint:
resp = client.messages.create(
model="claude-opus-4-8",
system=[{
"type": "text",
"text": LONG_STABLE_INSTRUCTIONS, # no timestamps in here!
"cache_control": {"type": "ephemeral"}, # cache everything up to this point
}],
messages=[{"role": "user", "content": todays_question}], # the only thing that varies
max_tokens=1024,
)
For OpenAI-family models (GPT-5.6 / 5.5) it's automatic above ~1024 tokens — same discipline, no breakpoint: keep the static stuff at the front and let it match the longest common prefix. Two things differ from Claude, worth knowing: there's no write premium (OpenAI doesn't charge to create the cache), and the read discount is smaller than Claude's 0.1x — you still save, just not 90%. You'll see it land in prompt_tokens_details.cached_tokens.
I moved the timestamp to the user message, left the system block untouched between calls, and re-ran. cache_read_input_tokens lit up on the second call and the bill for that agent dropped hard — because it loops twenty-plus times over the same head, and now that head is written once and read twenty times instead of written twenty times and read never.
Where caching actually pays (and where it doesn't)
It's amortization, not magic. It wins when a big, identical prefix is read many times:
Agent / tool loops — every step resends the same system prompt and tool schemas. Ideal.
Long-doc Q&A — paste the document once, ask ten questions against it.
Bulk extraction — fat instruction + few-shot block, only the input row changes.
It does nothing for one-shot calls where the whole prompt is different every time — there's no repeated prefix to amortize against. On Claude, explicitly caching there just burns the 1.25x write for no return; OpenAI won't charge you for the attempt, but it won't help either. Match the tool to the workload.
One gateway gotcha, since I run through one
I don't call the providers directly — I route through a gateway (I use . But the timestamp story above is really the whole lesson.
SOCIAL SHARE CARD GENERATOR