Anthropic's prompt caching is one of the highest-ROI LLM cost-reduction techniques shipped in the last two years, but the mechanics aren't immediately obvious from the docs. The pricing is non-uniform — a write premium on first writes balanced against a 90% discount on reads — and the marker syntax requires explicit opt-in rather than firing automatically the way OpenAI's does. The summary: tag the stable portion of your prompt with cache_control: { type: "ephemeral" }, pay 1.25x normal input price on the first request (5-minute TTL) or 2x (1-hour TTL), then 0.10x on every subsequent request within the cache TTL. Break-even on the 5-minute TTL arrives at the second cache hit; the 1-hour TTL takes a few more hits to pay back but survives much longer between requests. For most production workloads with a system prompt over a few hundred tokens, the discount kicks in by the second customer interaction. This post walks through the mechanics, the math, the gotchas, and the production patterns that turn the marker into actual savings.
The parent guide — get the boundaries right or the cache hits stop landing.
Production patterns
The shapes that hold up in production:
Stable system prompt + dynamic context + user message. The most common pattern. System prompt and tool definitions go in cached blocks; retrieved context and user message stay uncached. Almost every production LLM workload looks like this.
Two-level caching (system alone + system+context). When retrieved context changes per request but reuses a stable system prompt, mark both blocks for caching. The inner system-only cache still hits even when the outer system+context cache misses. Recovers a meaningful chunk of the saving.
Cache-warming on cold start. If your workload has predictable traffic patterns (e.g. business-hours support chatbot), fire a single warm-up request at the start of the active window to populate the cache. The first real user request hits the warmed cache instead of paying the write premium.
Per-user/per-session caching for personalised prompts. Each user gets their own cached prefix (with personalised system instructions). The cache hits within a single user's session but misses across users. The write premium is real but pays back across the second + third message of any conversation.
The anti-patterns
Three patterns that look like they should work but undermine the cache:
Injecting timestamps into the system prompt. "You are responding at [timestamp]. [Instructions...]" The cache fingerprint changes per request. Cache never hits. Strip dynamic content from the cached portion.
Marking everything for caching. The cache key is everything up to and including the marker. If you mark the very last content block (the user message itself), the cache key includes the user message, which makes it effectively useless — every request has a unique user message, so the cache never hits twice.
Caching prompts shorter than ~few hundred tokens. The write premium is real and the per-token savings are small on short prefixes. Anthropic's cache is most effective on prompts over 1,024 tokens; the breakeven on smaller prompts is rarely worth the complexity.
When OpenAI's automatic prompt cache is the better fit
OpenAI's prompt caching engages automatically with no caller-side configuration. The discount is smaller (50% vs Anthropic's 90%) but the operational simplicity is real. The trade:
If your application is OpenAI-heavy → no work needed; the discount applies automatically on prompts ≥1,024 tokens.
If your application is Anthropic-heavy → adopt the cache_control marker discipline; the 90% discount is materially larger.
If your application uses both → set up both patterns. Most production gateways (Prism included) handle this transparently — markers passed through to Anthropic, cached_tokens read back from both providers.
The deeper comparison: .
Decision framework
If you're standing up Anthropic prompt caching on a production workload:
Identify your stable prefix. System prompt + static instructions + tool definitions. Sum the token count. If it's over ~500 tokens, the cache is probably worth setting up.
Choose your TTL. Default 5-minute for active production traffic; 1-hour extended for spaced-out batch or daily-cron workloads.
Attach the marker.cache_control: { type: "ephemeral" }on the final content block of the cached portion.
Verify hits. Readcache_read_input_tokensfrom the response usage block on the second and subsequent requests. Should be non-zero on cache hits.
Avoid the anti-patterns. No timestamps in the cached portion. Don't mark the user message itself. Don't bother caching short prompts.
Layer with response-level caching for full coverage. Prompt caching discounts the calls that go through; response caching avoids many of them entirely. Read . For the OpenAI equivalent: . For the broader fingerprinting discipline: — the stable-prefix toggle drives the provider-native passthrough projection.
FAQ
What's the exact write premium?
25% above normal input price for the standard 5-minute TTL. The 1-hour extended TTL has a higher premium (confirm against Anthropic's current pricing page; pricing has moved historically). Both pay off within a small number of cache hits on most workloads.
Can I cache the user message?
You can, but it almost never makes sense. The cache key is everything up to and including the marker; if the user message is part of the key, the cache hits only on byte-identical user messages — which is rare in production. Mark the system prompt or tool definitions instead; let user messages stay uncached.
Does caching work with streaming responses?
Yes. The
streamparameter doesn't affect cache behaviour. Thecache_read_input_tokensandcache_creation_input_tokensappear in the final usage chunk of the stream (withstream_options.include_usageset). Streaming and prompt caching are independent.
What happens if I change the system prompt — do I have to invalidate the cache manually?
No. The cache fingerprint includes the system prompt content; any change automatically generates a different cache key, so old entries are unreachable for new requests. Old entries age out via TTL. No manual invalidation needed.
Can I use prompt caching with function calling?
Yes — and tool definitions are commonly part of the cached prefix. If your tools array is stable across requests, mark it for caching; the cache hits on the tool definitions even when user messages vary. Changing tools invalidates the cache for the affected requests.
Does the cache work across different models?
No. Cache entries are per-model. A request to
claude-opus-4-7doesn't hit cache entries fromclaude-sonnet-4-7. If you route between models per request (e.g. via a gateway like Prism), each model's cache warms independently.
What's the smallest prompt that benefits from caching?
Roughly 1,024 input tokens is the practical minimum where the cache machinery applies meaningfully — Anthropic's pricing and engineering are tuned for prompts at this scale and above. Caching a 200-token prompt is technically supported but the savings are negligible against the write premium and operational complexity. Use it on prompts that are actually long.
How does Prism handle this for non-Anthropic providers?
Prism passes provider-specific cache markers through to the target provider. OpenAI's automatic caching engages without markers; Anthropic's requires the cache_control attachment shown above. Customer code attaches markers explicitly; Prism doesn't auto-modify request shapes (with potential auto-marking opt-in for v1.9; see VERIFY tag above).
Anthropic's prompt cache is a real wedge on the right workloads. The lets you model the impact on your bill.
↗ Original-Artikel auf dev.to lesenVollständiger Original-BerichtAusführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.
SOCIAL SHARE CARD GENERATOR