KV Cache vs Prompt Cache: What's the Difference, and How Are They Related?

This article was originally published on my blog. For the latest version and future updates, please visit the original post: https://jaketao.com/language/en/kv-cache-vs-prompt-cache/. Every time a large language model generates a token, it draws on the content that came before it. If it had to compute everything from scratch at every step,...