Ai

Three recent papers all claim "cross-model" KV transfer. Each smuggles in a crutch that narrows what the word means. Here is the bar that would make the strict claim falsifiable. Cross-model KV transfer has a precise, seductive claim: one language model does the expensive prefill, another model decodes from the resulting state, and the second model never re-reads the prompt. If it worked, context would stop being a disposable token sequence and become a transportable artifact

What I measured. What I learned. What it means for KV cache design. In 2026, engineering teams are waking up to LLM bills that rival their cloud infrastructure. A significant portion of that cost isn't going into new work — it's going into reprocessing the same context, turn after turn, conversation after conversation. I wanted to measure exactly how much. So I configured Claude Code to point to my own inference runtime, with Qwen2.5-32B as the backend, and opened a session.

The token is not a choice. It's a hardware constraint — and it's the wrong level of abstraction for thinking about what a model understands. Imagine opening a book and having to read it from the beginning every time. Not because you forgot — because someone rips out your bookmarks between each reading. That is what a language model does. On every call, it rebuilds its understanding of the context from scratch. The state it produced — what it had "understood" — is thrown away