What counts as evidence in cross-model KV transfer
- Jul 7
- 6 min read
Three recent papers all claim "cross-model" KV transfer. Each smuggles in a crutch that narrows what the word means. Here is the bar that would make the strict claim falsifiable.
Cross-model KV transfer has a precise, seductive claim: one language model does the expensive prefill, another model decodes from the resulting state, and the second model never re-reads the prompt. If it worked, context would stop being a disposable token sequence and become a transportable artifact. Prefill cost would detach from the model that consumes its output.
The KV-First framework names this frontier explicitly. It defines a cross-model compatibility level (L2) — strict cross-model — by a single invariant: the receiver decodes only from a cache produced by a different model, the models are frozen, the architectures are heterogeneous, and the transfer is runtime-only — the target model's own prefill is never in the candidate path. That invariant is what "cross-model" has to mean for the claim to be a claim about transfer rather than about something easier. The framework does not solve L2; it states it.
The idea is recent enough that the evidence base is still being written, and the word "cross-model" is doing a lot of quiet work. In the last year, three systems — C2C, DroidSpeak, and PrefillShare — have each published a result framed as cross-model or cross-LLM KV sharing. Each reports a positive result. Each is also, on inspection, narrowing the definition of "cross-model" with a crutch that relaxes one of strict cross-model's constraints and does the hard part for it.
This is not an accusation of dishonesty — each paper states its scope. The problem is that the field has no shared definition strict enough to make a cross-model claim falsifiable, so the same label can mean transfer between two fine-tunes of one base, or transfer with the target model's own prefill still in the path, or transfer between models co-trained to be compatible. Three different claims, one word — and strict cross-model is the one none of them makes.
The question this essay asks is narrow: what would a strict cross-model claim have to report to count as evidence rather than a proxy? Answering it requires reading what each paper actually claims, naming the crutch each one rests on, and stating the evaluation discipline that strict cross-model demands and the literature has not built.
Three claims, three crutches
C2C — cross-LLM, with the receiver's own prefill kept in the path
C2C (Cache-to-Cache, Tsinghua, ICLR 2026) is the most genuinely cross-model of the three. Its title-level claim is cross-LLM communication — project one model's KV cache into another's space and let the second continue — and it backs it with a real cross-family pair (Llama-3.2 → Qwen3), a reported +8.5–10.5% accuracy gain, and seven pre-trained fusers published on HuggingFace.
The crutch is in the path, and C2C's own Table 8 names it. The ablation labels a "Project" configuration that directly replaces the receiver's KV cache with the projected one — the pure cross-model case — and shows that retaining the receiver's own cache via a residual connection lifts accuracy by 24.18%. The headline gain is measured on a path that keeps the target model's own prefill in it. The fuser does not have to carry the prompt alone; the receiver's own reading of the prompt is still there.
So C2C claims cross-LLM transfer and demonstrates cross-family projection. But the number that moves is the one where the target model also reads the prompt. The pure-replacement case — the one that would satisfy strict cross-model's no-target-prefill invariant — is the weak ablation, not the headline. C2C relaxes strict cross-model's last constraint.
DroidSpeak — cross-LLM, between fine-tunes of the same base
DroidSpeak (Microsoft, NSDI 2026) is titled "KV Cache Sharing for Cross-LLM Communication," and its scope is stated honestly: it reuses the KV cache between models fine-tuned from the same base model. Because they are only fine-tuned, they share the base's architecture, tokenizer, positional scheme, and most of their weights. DroidSpeak is training-free and recovers accuracy by recomputing the layers that diverged during fine-tuning.
The crutch is the definition of "cross-LLM" itself. A base model and its fine-tune are different checkpoints, but they are not heterogeneous models — they are the same architecture with different weights. The transfer is closer to a cache warm-start than a translation between distinct internal spaces. DroidSpeak's most useful move is to say so explicitly: cross-model is a spectrum, and DroidSpeak lives at the easy end. It does not claim, and does not test, the hard end where the two models differ in architecture. DroidSpeak relaxes strict cross-model's heterogeneity constraint.
So DroidSpeak claims cross-LLM communication and delivers it — for the narrowest reading of "cross-LLM," where the two models are fine-tunes of one base. The label is earned at the easy end of its own spectrum and not asserted beyond it.
PrefillShare — cross-model, between models co-trained to share a prefill
PrefillShare (2026) claims "cross-model prefill and KV-cache sharing" across "heterogeneous models," and the claim is real under its setup. It decouples each model into a shared prefill module and task-specific decode modules, freezes the prefill module, and fine-tunes only the decoders via cache-conditioned fine-tuning. Models that share the frozen prefill module can reuse one another's KV cache at inference.
The crutch is structural and is the whole mechanism. The models are made cross-model-compatible by construction: they are trained to share a prefill, then they share it. "Heterogeneous" refers to the fine-tuned decode side; the prefill side — the part whose KV cache is being shared — is identical by design. This is a training-time solution that removes the runtime problem by requiring access to training and by changing what the models are. PrefillShare relaxes strict cross-model's frozen-and-runtime constraints.
So PrefillShare claims cross-model KV sharing and achieves it — but on models it was allowed to reshape, not on frozen models it had to project between. It concedes the frozen-models, runtime-only case by solving a different one.
The bar
Each paper makes a legitimate, smaller claim under its own name. None makes the strict cross-model claim — the strict case is not contested by these papers; it is unclaimed. An unclaimed frontier has no evaluation apparatus attached to it, and without the apparatus, a claim that did arrive could not be checked.
Strict cross-model is the invariant; the bar states what a claim that holds to it would have to report. The first condition is the invariant itself, restated as something to evaluate against. The other three are the evaluation discipline the literature has not built.
No target prefill in the candidate path. The receiver decodes only from the transformed cache. The target model's own prefill may appear as a teacher or as an evaluation oracle; it may never appear in the path whose accuracy is the claim. This is the single test that separates strict transfer from the residual crutch — and C2C's Table 8 is the honest ablation that shows what happens when you remove it.
Delta against the native baseline, across task families. The claim is `score(receiver on transformed cache) − score(receiver on native prefill)`, reported per family, with deterministic decoding. At least three families that stress different capabilities — free-generation reasoning, multiple-choice QA, broader knowledge — so a method cannot survive only where the evaluation is friendly. The delta is the unit of truth; absolute scores are orientation. A fuser that lifts a weak receiver from 20% to 26% is not a win if the receiver's own native baseline is 40%.
Held-out disjoint split. Report the candidate's score on prompts the fuser never saw during training, alongside the score on the training surface. A result that holds on train and collapses on held-out is data-starved, not a map. This is the difference between "the fuser works" and "the fuser was trained on enough of the distribution to look like it works." This point bites hardest on the learned-projector strand — C2C and its successors — which is the strand that could plausibly reach strict cross-model.
Separation from perturbation controls. The real transformed cache must beat a cache of the wrong prompt, a shuffled cache, a pooled cache, and a zeroed cache. If it does not separate from these, the fuser has learned a shortcut, not the prompt. The control is the falsification mechanism: it is what makes the protocol able to say no.
A result that reports all four and still shows a non-negative delta has cleared a bar the current literature has not defined, let alone passed.
Why the bar is the contribution
Naming the bar is the contribution, because it separates three things the label "cross-model" currently conflates. A result with the target prefill in the path is a residual-fusion result, not a cross-model transfer result. A result between fine-tunes of one base is a same-architecture reuse result, not a heterogeneous transfer result. A result on co-trained models is a training-time compatibility result, not a runtime projection result. Each is worth publishing under its own name. None is strict cross-model.
L2 is the frontier KV-First names but does not solve: same-model compatibility is settled, cross-model compatibility is open, and the open is defined by the invariant stated up front. A frontier is not a settled feature. Saying what would settle it — the four-part bar above — is the work, because it tells the next experiment exactly what it must report and what it must disprove. The framework supplies the definition; the bar supplies the falsifiability.
The bar is not high to be cautious. It is high because every lower bar has already been crossed by a result that kept a crutch in the path and called it cross-model.
Comments