2605.28713v1 May 27, 2026 cs.AI

Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor

Daiting Shi
Daiting Shi
Citations: 297
h-index: 7
Zhiyuan Sun
Zhiyuan Sun
Citations: 3
h-index: 1
Yuci Liang
Yuci Liang
Citations: 53
h-index: 3
Guoxin Ma
Guoxin Ma
Citations: 23
h-index: 2
Yibin Liu
Yibin Liu
Citations: 59
h-index: 4
Ke Chen
Ke Chen
Citations: 20
h-index: 3
Chengzhengxu Li
Chengzhengxu Li
Citations: 69
h-index: 5
Yan Wang
Yan Wang
Citations: 12
h-index: 2
Zhaohan Zhang
Zhaohan Zhang
Citations: 14
h-index: 2
Yue Zhang
Yue Zhang
Citations: 79
h-index: 3

Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown promise, they typically rely on complex compression modules or compression-specific training, leaving the intrinsic capabilities of LLMs underexplored. In contrast, this work reveals that a thinking model itself can naturally compress long contexts by organizing task-relevant information. We thus derive Thinking as Compression (TaC), a new compression paradigm that treats thinking itself as compressed context. Without relying on specific dedicated compressor, TaC directly prompts the thinking model to generate thinking traces as the shortened context, already outperforming most representative compression methods. Further, given that raw thinking output may struggle with budget control and shortcut behaviors, we introduce Thinking as Compression Constrained (TaC-C), leveraging a simple reward-driven optimization framework to elicit intrinsic thinking as compact and controllable compressed context. Experiments across four long-context QA benchmarks demonstrate that TaC-C consistently outperforms existing baselines. At 4x and 8x compression ratios, it surpasses the strongest competitor by 17.4% and 23.4% in average F1, and by 15.7% and 21.7% in average Exact Match Score (EM), respectively.

0 Citations
0 Influential
3.5 Altmetric
17.5 Score
Original PDF

No Analysis Report Yet

This paper hasn't been analyzed by Gemini yet.

Log in to request an AI analysis.

댓글

댓글을 작성하려면 로그인하세요.

아직 댓글이 없습니다. 첫 번째 댓글을 남겨보세요!