Cache-to-Cache

Direct semantic communication between LLMs through KV cache projection and fusion, with higher accuracy and lower latency than text-based communication.

+11.0 pp
Accuracy gain vs Receiver
+5.4 pp
Accuracy gain vs T2T
3.46×
Latency speedup vs T2T
Read Paper View Code 🤗 Models ICLR 2026

Overview

LLMs communicating beyond text

Multi-LLM systems combine diverse models to leverage their complementary strengths. But current text-based communication creates bottlenecks.

Text-to-Text (T2T) loses rich semantic information and requires slow token-by-token generation
Cache-to-Cache (C2C) fuses the Sharer’s KV cache into the Receiver’s, sharing contextual understanding without intermediate text generation.
C2C Overview

Motivation

Why communicate beyond text?

Text-based communication between LLMs has fundamental limitations

Information Bottleneck

High-dimensional semantic representations are compressed into linear text sequences

Ambiguity

Natural language is inherently vague and can be misinterpreted by the receiver

Latency

Slow token-by-token generation is required for every communication exchange

T2T vs C2C Example

Example: Text-to-Text communication loses structural semantics through ambiguous descriptions. Cache-to-Cache transfers precise semantic understanding directly.

The Discovery

KV Cache carries semantic meaning.

Oracle experiments reveal three key findings that motivate Cache-to-Cache communication.

Beneficial
Cache Enrichment Works
Enriching KV cache semantics achieves near few-shot performance (62.3% vs 63.4%) without extending sequence length.
Convertible
Cache Transfers Across Models
KV cache from one model can be projected into another model's representation space via simple transformation.
Complementary
Models Have Distinct Strengths
Different LLMs encode distinct semantic understandings. Their correct-answer sets show limited overlap despite similar accuracy.

Oracle 1: Cache Enrichment

Question: Can enriched KV cache improve response quality without increasing cache size?

Does few-shot prompting improve accuracy by attending to more tokens, or by enriching how the question is embedded in KV cache? We test this by prefilling on exemplars E⊕X, then discarding E and keeping only the question-aligned cache slice.

  • Direct: 58.42% accuracy (cache length |X|)
  • Few-shot: 63.39% accuracy (cache length |E| + |X|)
  • Oracle: 62.34% accuracy (cache length |X| only)

Finding: The gain comes from richer question embeddings, not from attending to extra tokens. KV cache can carry enriched semantics.

Oracle 2: Cache Transformation

Question: Can one model's KV cache be utilized by another model?

We trained a 3-layer MLP to map KV cache from Qwen3-4B (source) to Qwen3-0.6B (target). T-SNE visualization shows the transformed cache moves into the target model's representation space.

t-SNE visualization of cache transformation

Finding: The transformed cache occupies a subset of the target's space, reflecting distinct semantic understandings across models. This suggests fusing specialized contextual understanding from different models could harness their complementary strengths.

Architecture

Cache-to-Cache Fuser

C2C Architecture

C2C Fuser: Projects and combines both models’ KV caches, then adds the result to the Receiver’s cache through a learnable gate.

Projection Module

Concatenates and processes KV caches from both models through projection and feature fusion layers

Dynamic Weighting

Input-aware head modulation layer for adaptive information flow between models

Learnable Gating

Trainable per-layer gates control fusion; input-aware weights modulate how much information is added.

Training: Freeze both models, train only C2C module with next-token prediction loss

Experiments

Benchmark Results

Qwen3-0.6B Receiver with three Sharers across four benchmarks.

Headline metrics: Qwen3-0.6B Receiver · Qwen2.5-0.5B Sharer · four-benchmark average (Table 4).

Accuracy gains are percentage-point differences. Speedup is mean T2T latency divided by mean C2C latency over the selected rows. Single A100, batch size 1; paper Table 4.

—
Accuracy gain vs Receiver (pp)
—
Accuracy gain vs T2T (pp)
—
Speedup vs T2T
View Detailed Results
Loading results...
Receiver Sharer Benchmark Receiver Only Sharer Only Routing Text-to-Text Cache-to-Cache

Summary

Key Contributions

Novel Paradigm

Direct semantic communication through learned KV cache projection and residual fusion.

Effective & Efficient

With Qwen2.5-0.5B as Sharer, C2C improves Qwen3-0.6B accuracy by 11.0 percentage points and cuts T2T latency by a factor of 3.46, averaged across four benchmarks.

Flexible & General

Supports different model families, sizes, and specializations through trained fusers with token and layer alignment.

Team

Meet the researchers.

Tianyu Fu1,2* Zihan Min1* Hanling Zhang3* Jichao Yan1 Guohao Dai5,2 Wanli Ouyang3,4 Yu Wang1†
1Tsinghua University    2Infinigence AI    3The Chinese University of Hong Kong
4Shanghai AI Laboratory    5Shanghai Jiao Tong University
*Equal contribution    †Corresponding author

Citation

Cite our work.

If you find C2C useful for your research, please consider citing our paper.

BibTeX 
@article{fu2025c2c,
  title={Cache-to-Cache: Direct Semantic Communication Between Large Language Models},
  author={Fu, Tianyu and Min, Zihan and Zhang, Hanling and Yan, Jichao and Dai, Guohao and Ouyang, Wanli and Wang, Yu},
  journal={arXiv preprint arXiv:2510.03215},
  year={2025}
}