AWS explains prompt caching for Bedrock agents

AWS’s implementation guide addresses agents that repeatedly send instructions, tool definitions and conversation history to the model. Its practical starting point is an unchanged beginning: keep instructions and tool serialization stable, then put changing dates, names or request identifiers afterwards. In its example, disabling response storage leaves history management to the application; it does not turn off prompt caching.

OpenAI’s caching documentation explains why this differs from remembering a finished answer: the cache preserves computed model state for the unchanged beginning of a request. When a matching entry is available, that work can be reused, while new input still needs processing. For conversations, its advice is to append messages and retain earlier messages and tool results. Summarizing or truncating earlier context can change the prefix and interrupt reuse. The model still generates a fresh response, so identical requests do not guarantee identical answers.

The Bedrock reference explains the controls for GPT-5.6 and later: the default implicit mode chooses a cache boundary automatically and also accepts developer-supplied boundaries. Explicit mode uses only the supplied markers; without them, it does not cache. The reusable beginning must meet the model’s minimum input length, and each request can create at most four cache writes; support varies by model and API. To check actual reuse, inspect cached_tokens for reads and cache_write_tokens for writes, then compare their volumes: eligibility alone does not guarantee a hit.