cached_tokens is 0 because your system prompt isn't stable

I added a [cache] log line next to our usage tracker and read usage.prompt_tokens_details.cached_tokens off a warm conversation. It said 0. Then 1. Out of roughly 16,000 prompt tokens, every single turn. We were paying full price for the same prefix over and over, and the 273K-token board balloon I'd been blaming on the agent loop was just real... Weiterlesen