As coding Agent tasks become increasingly complex and execution chains grow longer, token overhead is surging. Effectively controlling token costs has become an engineering challenge that must be addressed for AI to scale.
Open-source repository: github.com/aixcoder-plugin/aiXcompact-4B
On November 11, aiXcoder, incubated by the Institute of Software Engineering at Peking University, open-sourced aiXcompact-4B, a “dedicated context compression model” designed for long-horizon, complex R&D tasks involving coding Agents and supporting private deployment on a single A100-40GB GPU.
According to data published by aiXcoder, in the Terminal-Bench benchmark, testing context compression performance by connecting aiXcompact-4B to OpenCode showed no decline in overall task completion accuracy compared with DeepSeek-V4-Pro. In private deployment scenarios, on task-specific benchmarks, aiXcompact-4B reduced single-GPU inference costs by 78% compared with DeepSeek-V4-Pro.
Unlike mainstream Agent Harnesses (agent execution frameworks) such as Claude Code and OpenCode, which directly call general-purpose large models to perform context compression, aiXcompact-4B takes over this high-frequency workload at lower cost: it condenses the historical context continually accumulated by the Agent, preserves the information needed for subsequent tasks, and reduces the token overhead of both the compression itself and repeated inputs in subsequent steps.
When invoked via API, based on proxy estimates derived from public API fees, the token cost of a single context compression request with aiXcompact-4B is RMB 0.019. Compared with the reference costs of DeepSeek-V4-Pro, GLM-5.3, and Qwen3.8-Max, this represents reductions of 97.30%, 97.08%, and 97.98%, respectively. With its 97% lower cost per API call, aiXcompact-4B provides a lighter and more economical option for high-frequency context compression.
After context compression, the historical context that the Agent must repeatedly read is reduced accordingly. In target-task scenarios, aiXcompact-4B compresses historical context to an average of 25.89% of its original length, freeing ample context space for the next Agent call.
This open-source release demonstrates a more granular division of labor among models: assigning high-frequency, clearly bounded context compression tasks to a specialized small model, while allowing general-purpose large models to focus on requirement decomposition, complex reasoning, architecture design, and similar tasks. This reduces resource consumption in auxiliary steps while ensuring task execution quality.
An average token consumption of 4.17 million—why are coding Agents so “token-hungry”?
As AI coding tools such as Claude Code and Codex are used for increasingly complex R&D tasks, Agents must continuously read code, invoke tools, and run tests, driving token costs higher.
A study based on the OpenHands framework and covering eight frontier models found that, in SWE-bench Verified experiments, agentic coding tasks consumed an average of approximately 4.17 million tokens—3,500 times the consumption of single-turn code reasoning and 1,200 times that of multi-turn code conversations—with input tokens accounting for the vast majority.
This is closely tied to how Agents work. For example, adding user registration functionality to an existing enterprise system requires the Agent to first understand the project structure and locate the user management and authentication logic, then modify files, run tests, and continue adjusting based on the errors. Throughout this process, code, retrieval results, and execution logs continuously enter the context and become the basis for subsequent work. The same batch of historical information may be fed in repeatedly across multiple rounds of reasoning, and as execution steps increase, the token overhead from repeated inputs continues to accumulate.
Beyond costs, early requirements, critical constraints, and confirmed technical decisions are mixed in with a large amount of intermediate records, which may also increase the risk that the Agent will overlook important information or lose debugging clues.
Therefore, context compression has become an important engineering step for controlling Agent operating costs and supporting the continuous execution of long-running tasks. It preserves key information and streamlines redundant content from the Agent’s continuously growing execution trajectory, reducing the token overhead of repeated input and leaving more context space for the current code, the latest feedback, and subsequent decisions, enabling the Agent to complete complex, long-horizon R&D tasks with higher quality.
aiXcompact-4B Structured Compression to Reduce Redundant Token Overhead in the Context
The context of a coding Agent has clear roles and a structured flow: the developer presents requirements, the Agent analyzes the problem and calls tools, and the tools return file contents or execution results. For an Agent that needs to execute tasks continuously, too little compression saves little context space, while too much compression may lose key steps from the history and damage the structured information in code and tool calls.
Effective compression must both understand the content of the information and preserve the relationships among those contexts. According to the aiXcompact-4B open-source project, the model uses a structured discrimination mechanism to generate three types of handling decisions for historical trajectories.
- First, preserve critical original information.Information such as the developer’s core requirements, key constraints, and major decisions may determine the direction of subsequent code changes and should not be arbitrarily removed. aiXcompact-4B chooses to preserve original messages in full, reducing the risk that key qualifiers will be weakened during rewriting.
- Second, streamline tool-returned results.File reads, code searches, and test runs often return large volumes of content, but the next step may require only a portion of it. aiXcompact-4B preserves the tool calls and replaces verbose raw outputs with condensed text, reducing input tokens while retaining the clues needed to continue diagnosing problems and modifying code.
- Third, remove records that have lost their reference value. For repetitive content or intermediate output that is no longer needed, aiXcompact-4B removes it and performs corresponding validation actions to check whether key business and debugging information has been retained in the task summary.
After completing information selection, the model also generates a structured task summary recording the user’s intent, key technical information, errors and their resolution processes, current work, and remaining tasks. The final output is a sequence of messages that the Agent can use to continue execution while maintaining the pairing between tool calls and their returned results.
From the perspective of its operating principles, aiXcompact-4B does not simply shorten the context or condense it into a single summary. It distinguishes information that must not be lost, information that can be streamlined, and information that is no longer necessary, while also ensuring that the compressed context remains executable. Its core objective is to reduce redundant token consumption while preserving the context and structure necessary for the Agent to complete subsequent tasks.
Trained onReal R&D Trajectories,the Small Model’s Compression MetricsApproachDeepSeek-V4-Pro4B
How does a 4B-parameter model learn to judge the value of information in complex R&D records?
This capability is supported by aiXcompact-4B’s training approach, which combines teacher-model distillation, LoRA fine-tuning of a 4B student model, and structured compression decisions.
According to aiXcoder, all of its training data comes from Agent execution trajectories in real R&D scenarios. The team first normalizes the trajectories, filters and selects them, and samples compression trigger points before having the teacher model generate reference compression decisions.
After these reference results undergo sample validation and filtering, they are used for specialized student-model training, enabling the model to learn how to process historical context in conjunction with R&D progress and how to preserve the basis for subsequent work in the task summary.
In terms of the division of technical responsibilities, distillation transfers the teacher model’s processing capabilities for this task to the specialized student model, while LoRA fine-tuning adapts the model to the compression scenario. Structured decisions further constrain the form in which this capability is expressed, allowing it to integrate with the Agent’s messaging and execution processes.
In aiXcoder’s published results for the dedicated context compression evaluation, aiXcompact-4B closely approached the historical benchmark performance of DeepSeek-V4-Pro on key metrics such as the composite rule score and information retention.
More granular division of labor between large and small models, tailored to enterprise-grade private deployment and engineering implementation,adapted toenterprise-grade privatedeployment and engineering implementation
As enterprise intelligent software development enters increasingly challenging territory, the requirements for model capabilities across different R&D stages have become increasingly differentiated, creating room for a more granular division of labor among models. Enabling different models to take on tasks suited to them, while maintaining completion quality and reducing Token consumption and improving computing-resource utilization efficiency, is an engineering challenge that enterprises must continually address when implementing AI-driven R&D.
In response, aiXcoder introduced a collaborative “large model + small model” solution: large models focus on requirements decomposition, complex logical reasoning, and architectural design, while specialized small models handle specialized work with relatively clear boundaries and high invocation frequency. This reduces enterprises’ computing costs and addresses the practical challenge of high computing costs when implementing AI-driven R&D.
It is worth noting that aiXcompact-4B is not the first specialized small model introduced by aiXcoder. aiXcoder continues to uphold the concept of “scenario-defined models,” building a matrix of specialized models for business scenarios and specific task requirements across different domains. In software development, aiXcoder has introduced models such as aiXcoder-7B for code generation and completion, aiXapply-4B for code change applications, and aiXpruner-4B for Token consumption optimization. In addition, aiXcoder is about to launch the aiXdecision model for Agent decision-making, focusing on judgment and decision support during agent execution and further expanding the application boundaries of specialized models.
By open-sourcing aiXcompact-4B, aiXcoder has further expanded its model matrix to Agent context management, reflecting its technical approach of defining model capabilities based on concrete scenarios and task requirements and supporting complex tasks across different domains through collaboration between large and small models.
