AKV gives AI agents control over their computation cache

The AKV project addresses a growing burden for AI agents: longer conversations increase the storage and processing needed for their remembered computation. This stored state is called the KV cache. AKV lets an agent discard selected states while reusing compatible retained ones, without deleting the corresponding text. Dropping a state from one request does not immediately free shared storage: another active computation or restoration may still need it.

The repository’s example keeps the complete conversation in the request. A policy adds instructions for dropping cache entries and compacting the remaining positions. After processing a newer tool result, the server drops the earlier result’s cache, compacts positions and generates the next reply. Message numbering stays unchanged. A separate demonstration lets the model request drops through a tool; the policy converts successful calls before the next model request.

The system documentation makes this a deployment task. Installation uses Rust, Cargo and a Python 3.12 environment; startup compilation also needs a working C++20 compiler and writable compilation cache. Support covers specified Qwen, GPT-OSS and MiniMax-M2 text architectures. Cache-operation requests currently exclude features including speculative decoding, LoRA adapters and external or session caches. Unsupported requests are rejected; successfully starting a model does not establish compatibility with these cache operations.