CliffCompaction: Cost-Efficient Context Compaction for Long-Horizon Coding Agents
CliffCompaction：長任務程式碼 Agent 的高效能 context 自動壓縮技術
As coding agents tackle complex, million-token tasks, context window limits and soaring costs pose major bottlenecks. CliffCompaction introduces an autocompaction method that strictly truncates or drops content without rewriting, and never compacts an already-compacted context to avoid context drift. It cuts costs by up to 50% while boosting agent performance. Under parallel test-time scaling, it enables cheaper models (like Kimi K2.6) to match or exceed flagship models (such as GPT-5.3 Codex) at a fraction of the cost.
Key points
Truncation-Only Faithfulness
Keeps compacted information faithful by only truncating or dropping content, never rephrasing or rewriting it to eliminate LLM hallucinations.
No Double Compaction
Never compacts a compaction; each pass operates only on original content and discards prior compacted output to prevent compounding context drift.
Efficient Test-Time Scaling
Reduces per-rollout costs to make test-time scaling more efficient, adding over 10 percentage points on Terminal-Bench for less than the cost of two full runs.
Million-Token Continual Learning
Sustains performance over sessions exceeding a million tokens, reaching 3.58x CUDA kernel speedups after 400 steps on KernelBench.
How it works
| 傳統 LLM 重寫壓縮 (Traditional Rewriting) | CliffCompaction 自動壓縮 | |
|---|---|---|
| Core Method | 透過 LLM 重新撰寫、生成摘要 | 僅進行截斷與捨棄 (Truncate / Drop) |
| Faithfulness | 較低，LLM 重寫可能引入幻覺與語意扭曲 | 極高，保留內容完全與原始碼/終端機輸出一致 |
| Context Drift | 嚴重，不斷對「已壓縮的內容」再次壓縮造成誤差累積 | 無，每次直接操作原始內容並丟棄舊壓縮輸出 |
| Long-Horizon Scaling | 受限，隨步驟增加脈絡品質快速衰退 | 極佳，支援百萬 token 且在 400 步後仍持續學習 |
Why it matters
This research tackles the core bottleneck of deploying long-horizon agents: prohibitive token costs and context degradation. By demonstrating that selective truncation outperforms LLM-based rewriting, CliffCompaction prevents hallucination and context drift. It offers developers a scaffold-agnostic, cost-effective API proxy to scale existing coding models (like Claude Code or Codex) on massive codebases with minimal overhead.
Who it affects
- AI Developer
- AI Researcher
- Enterprise Leader
How to use it
- 1Optimizing token consumption and context accuracy for long-horizon coding agents (e.g., Claude Code) on massive codebases.
- 2Accelerating continuous learning tasks like CUDA kernel optimization over hundreds of sequential trail-and-error steps.
- 3Implementing cost-effective test-time scaling to allow mid-tier models to match flagship models via parallel rollouts.
Limitations & caveats
- Since it strictly truncates rather than synthesizes, it may perform less optimally on general prose tasks that demand deep semantic integration.
- The technique has been primarily validated on software engineering and terminal-based tasks, meaning general-domain applicability is still unproven.
Related
SWE-Serve: Benchmarking Agentic Engineering for Production Inference Serving
SWE-Serve：首個針對「生產級推論服務」的 AI Agent 軟體工程基準測試
SWE-Serve is a new benchmark featuring 53 real-world SGLang tasks, designed to evaluate AI agents' capability to implement complex features and achieve production correctness across the inference serving stack.