7 min read
How the KV cache shapes AI agent design
Prompt caching only works when each request begins with the same tokens as the last. The design rules that follow for AI agents, and how to check them.
Articles on KV cache
1 article, newest first.
1 article, newest first.
7 min read
Prompt caching only works when each request begins with the same tokens as the last. The design rules that follow for AI agents, and how to check them.