Yunzhong, from Aofeisi Temple

QbitAI | QbitAI’s official WeChat account

Why does an Agent get stronger the more you use it, yet still get “tripped up” by the environment?

Your colleague uses an Agent to organize reports and sails smoothly all the way. You use the same model, tools, and workflow, but with your own files, the Agent starts reading them repeatedly and reworking the results constantly.

The model has not changed, and neither has the Harness. Why does it stop working in a different environment?

Different file locations, mixed data formats, and changes in template constraints can all cause methods that once worked to fail. The model provides reasoning capabilities, and the Harness organizes the execution flow, but the Agent still has to determine: what is this environment like, and do past experiences still apply?

This also exposes an easily overlooked issue in the Agent’s continuous self-improvement:

an Agent not only has to learn “how to do it” from a task, but also determine under what environments those experiences remain valid.

During the continuous optimization of RSI (Recursive Self-Improvement), an Agent can improve its execution methods through task feedback and accumulate reusable experiences. But if its understanding of the environment is not updated in sync, previously effective experiences may become new obstacles: the interface conditions have changed, yet it keeps retrying old calls; the data format has changed, yet it keeps following old rules.

After a failure, should the method be changed, or should the understanding of the environment be corrected?

The Agent’s new challenge contains an old clue from cognitive science

Research on knowledge structures in cognitive science offers inspiration for distinguishing between these two types of problems:

In 1983, cognitive scientist John R. Anderson systematically discussed two important forms of human knowledge in his research on ACT-R-related theories:

• Declarative Knowledge: Describes “what the world is like.” For example, a closed container obscures the objects inside, and they can only be observed after it is opened.
• Procedural Knowledge: Describes “how one should act.” For example, when looking for multiple objects of the same kind, after finding the first one, one should continue checking other possible locations.

Bringing this perspective to the Agent reveals that:understanding the environment and mastering the methods are interrelated, yet require different forms of validation and updating. Environmental judgments must be checked against the facts on site, while action methods must be tested for effectiveness under the corresponding conditions. Confusing the two may cause an operation that succeeded by chance to be treated as a stable pattern, or cause an invalid method to continue being used after the environment changes.

Building on this idea, in its continued exploration of RSI, openJiuwen further proposed TTSE (Two-Track Self-Evolution, dual-track self-evolution).

In the previously released introduction to the dual-dimensional RSI framework, openJiuwen had already enabled Agents to continuously improve their working methods and deliverables based on real task feedback through Harness+Artifacts dual-dimensional optimization. TTSE goes further by focusing on the reliability and reusability of experience itself, enabling Agents not only to accumulate effective execution methods but also to continually refine their understanding of the environment, preventing experience from becoming invalid after environmental changes.

To this end, drawing on the distinction between declarative and procedural knowledge in cognitive science, TTSE divides the experience acquired through tasks into two tracks:

• FACT (environmental facts): Continuously verify and update “what this environment is like.”
• TIP (conditional techniques): Continuously improve “what should be done under these conditions.”

The TTSE dual-track framework: Building a feedback-driven closed loop for the self-evolution of experience

Distinguishing the types of experience is only the first step.How can an Agent continuously learn from task feedback and turn effective experience into improvements for the next round of execution?

This is also the core idea of RSI: driving improvement through real task feedback and continually validating and optimizing it in subsequent executions. TTSE implements this idea at the experience-management level by executing tasks in batches, allowing existing experience to be continuously tested against new evidence, determining which experiences should be retained, revised, or retired, and then applying the updated experience to the next round of tasks.

The framework consists of four parts:dual-track knowledge representation, cold start, online incremental evolution, and final output and evaluation, as shown below:

① Dual-track knowledge representation: Grounding environmental facts and execution methods

FACT describes the attributes and regularities of the environment itself, emphasizing direct verifiability and cross-task stability in applicable scenarios; TIP describes execution methods under specific task conditions, expressed in the form “under what conditions, what operation to perform.”

The two types of knowledge maintain separate numbering, presentation formats, and evidence-tracking mechanisms. This allows each experience to be cited, checked, and updated independently, and makes it easier to trace exactly which factual judgment or operational recommendation had an impact when an execution goes wrong.

During classification, the framework makes judgments based on the subject and role described by the experience (details on how classification is performed based on the mechanism); content that is difficult to determine is assigned to TIP by default, reducing the risk of mistaking insufficiently validated operational experience for an environmental fact.

② Cold start: Deriving initial experience from the first batch of tasks

TTSE starts with an empty rule repository. The Agent executes the first batch of tasks (chunk 0), then separately derives FACT and TIP from the execution traces to establish an initial dual-track experience repository.

This stage focuses primarily on induction. Since no historical rules are yet involved in execution, old rules are neither held responsible nor retired; instead, the first batch of experiences backed by task evidence is established for subsequent evolution.

③ Online incremental evolution: Testing and revising existing experience in new tasks

Subsequent tasks proceed in batches. In each round, the Agent loads the FACT and TIP from the previous round and uses existing experience to execute the current batch of tasks. It then completes four stages in sequence based on the execution traces and evaluation results:

• Attribution (Blame): Analyze failed trajectories, identify rules that may have misled execution, and pinpoint targets for experience refinement.
• Retirement (Retire): Move rules deemed responsible into the retirement pool, retaining records of reasons, batches, timestamps, and other information, so that the adjustment process is traceable.
• Contradiction Analysis and Synthesis (Contradict / Synthesize): Examine contradictions among retired rules, attempt to identify their implicit applicability conditions, and synthesize seemingly conflicting experiences into a TIP with more clearly defined conditions.
• Induction (Induce): Compare successful and failed trajectories, or distill effective experience from successful trajectories, and update the dual-track experience repository through operations such as addition, revision, confirmation, and removal.

Blame→Retire→Contradict/Synthesize→Induce operates cyclically across multiple task batches, allowing experience to be continuously tested against new evidence: updating both the understanding of the environment and the execution methods under corresponding conditions.

④ Final Output and Evaluation: Assessing the Practical Gains Brought by Experience Evolution

After multiple rounds of incremental evolution, the framework outputs the final FACT and TIP experience repositories and evaluates their impact on accuracy or task performance in test tasks.

At the core of this mechanism is allowing environmental facts and execution methods to accumulate separately and remain independently traceable, so that they can be tested and refined through continuous execution. Each time an Agent completes a batch of tasks, it can transform new interaction evidence into experience usable in the next iteration, gradually forming a closed loop from environmental interaction to knowledge updating and then to action improvement.

“However, one key question remains to be answered: Is the separate evolution of environmental facts and action techniques truly more effective than traditional single-track experience management?”

From Mechanism to Empirical Validation: Five Benchmarks for Verifying the Effectiveness of TTSE Dual-Track Self-Evolution

The methodology and experimental results of TTSE are presented in the paper A Two-Track Online Self-Evolution Framework for LLM Agents.

To verify whether the dual-track experience mechanism is truly effective, the research team evaluated the method on five benchmarks: ALFWorld, ScienceWorld, GDPevo, SOPBench, and PinchBench. Each experiment was conducted three times independently to examine the method's practical gains and run-to-run stability.

Five Benchmarks Show Gains Across Multiple Dimensions

• ALFWorld: The success rate reaches as high as 98.51%.On GLM-5, TTSE achieves a success rate of 98.51%; on DeepSeek-V3.2, it reaches 97.8%, an improvement of 14.2 percentage points over the previously best-performing method, GraSP, at 83.6%, while also reducing the number of task execution steps.
• ScienceWorld: Task performance improves significantly.MACRO Reward increases from 0.7204 to 0.8534, and Native Success rises from 53.08% to 73.46%, an increase of 20.38 percentage points.
• GDPevo: The dual-track approach outperforms the single-track approach.The complete FACT+TIP solution achieves an average score of 0.2616, approximately 2.8 times the zero-shot baseline of 0.0947. Neither the TIP-only nor the FACT-only approach matches the performance of the complete dual-track solution, validating the value of jointly managing environmental facts and action techniques.
• SOPBench: Further improved after integration with Bayesian skill management. The overall success rate increased from the baseline of 79.15% to 91.59% with the single-track approach, and further to 94.58% with the dual-track fusion approach—an additional 2.99 percentage points over the single-track approach.
• PinchBench: Consistent gains maintained on real-world tasks. TTSE was integrated into OpenClaw non-invasively and evaluated over three independent runs on 147 real-world end-to-end tasks. The dual-track approach’s average score increased from 0.7439 to 0.7822, a gain of 3.83 percentage points; the direction of the gains was consistent across all three runs, and the range decreased from 3.33 to 1.61 percentage points.

These results cover embodied interaction, scientific reasoning, enterprise operations, industrial processes, and real-world end-to-end tasks, demonstrating that TTSE’s value is not limited to any particular model or single scenario.

Ablation studies further answer: why is a dual-track design necessary?

The ablation study on GDPevo provides more direct evidence: the complete dual-track approach outperforms single-track approaches that record only facts or only skills.

The reason is that environmental cognition and action strategies serve different purposes. With FACT alone, the agent may understand the environment’s constraints but lack suitable methods of action; with TIP alone, the agent may master operational techniques but be unable to accurately identify the conditions under which those techniques apply. Only by combining the two can the agent simultaneously answer “What is the environment like?” and “What should be done under the current conditions?”

Therefore, the dual-track design does not simply divide the experience repository into two parts; it enables different types of knowledge to be summarized, validated, and reused according to their respective semantics and applicability conditions.

From cross-model and cross-task evaluations to real-world evaluations, the experimental results not only validate the effectiveness of TTSE’s dual-track experience mechanism but also further demonstrate the potential of RSI in experience learning: without changing model parameters, an Agent can gradually improve its performance on subsequent tasks by continuously accumulating, validating, and updating task experiences.

This also provides empirical support for TTSE’s transition from a research framework to a practical office-agent product.

TTSE Dual-Track Self-Evolution Deployed in WorkSwarm

TTSE has not only demonstrated its effectiveness in paper experiments but has also been officially deployed in openJiuwen’s WorkSwarm swarm office agent.

Want to experience TTSE for yourself? Simply install and launch WorkSwarm version 0.2.7 or later and enable the TTSE self-evolution switch in the settings to experience the efficiency gains brought by experience accumulation and reuse.

Next, let’s explore through two real-world office tasks how TTSE enables Agents to execute tasks faster and more efficiently in practice.

TTSE Enters Real-World Office Scenarios, Turning Experience into Efficiency

In everyday office work, Agents are now widely used for tasks such as data cleaning and analysis, document editing and organization, report generation, information extraction, and format conversion. However, when faced with different file structures, data formats, and business requirements, Agents often need to repeatedly explore the environment, experiment with tool calls, and rediscover the task execution path.

To clearly demonstrate TTSE’s practical impact, the team selected two typical office scenarios—Excel data cleaning and document and spreadsheet organization—and compared the Agent’s performance on its first execution with its performance after experience injection.

The results show that both scenarios achieved varying degrees of efficiency improvement:

These improvements did not stem from changes in the model’s capabilities. Instead, in the comparative tasks shown, the Agent was able to draw on execution experience from previous runs to identify environmental constraints more quickly and select an appropriate way to handle them.

So, what experience has TTSE accumulated, and how does it help the Agent reduce trial and error?

Case 1: Excel Data Cleaning—Reducing Trial and Error in Data Identification and Processing

The difficulty of such tasks lies not only in cleaning the data, but also in locating input files, identifying hidden data-format issues, and determining deduplication rules that meet business requirements.

On the first execution, the Agent had no reusable task experience and had to independently locate the input files, determine the data format, and derive rules for handling duplicate records. The entire process involved57 tool calls, 3 of which failed; the total time was 6 minutes 22 seconds.

After the task was completed, TTSE distilled reusable task experience from the actual execution trajectory. This experience was no longer merely a record of past operations; it could directly inform decision-making in the next execution:

• Rapidly locate input materials.When the user does not specify file locations, check workspace/input first and directly locate upstream files, avoiding repeated exploration of unrelated task directories and eliminating unnecessary follow-up questions to the user.
• Identify data issues in depth.For dates, amounts, and text in Excel that appear normal, use openpyxl directly to inspect the underlying stored values and identify hidden issues such as text-formatted dates, currency symbols, and trailing spaces, reducing omissions caused by improper reading methods.
• Reuse validated deduplication rules.Group records by business fields, retain the record with the earliest date, and record the number of deletions. There is no need to derive the deduplication criteria from scratch again, reducing repeated judgment and trial and error.

When executing a similar task again, this experience directly informs tool selection and processing decisions. Tool calls dropped to20, failed calls dropped to 1, and the total time was reduced to 1 minute 48 seconds.

Case 2: Document and Spreadsheet Organization—Reducing File Exploration and Format Trial and Error

Unlike Excel data cleaning, the main challenges of such tasks lie in unknown file locations, unclear CSV encodings and delimiters, and complex Word template structures. On the first execution, the Agent had to gradually explore the files, identify their formats, and determine an appropriate approach to handling the template.

On the first execution of the document and spreadsheet organization task, the Agent took

1 minute 49 seconds, made 27 tool calls in total, and had 2 tool-call failures. After the task was completed, TTSE distilled a set of environmental facts and operational techniques from the actual execution trajectory.

On the second execution, the Agent no longer had to explore the file directory from scratch, test different ways of reading CSVs, or wait until after the document had been generated to discover template replacement issues. It could directly reuse its validated knowledge of the environment and select a more appropriate tool and processing path in advance.

The second execution time was reduced to

1 minute 26 seconds, tool calls fell to 18, and failures fell to 1. Overall execution time was reduced by approximately21.1%, tool calls by33.3%, and failed calls by50%.

Rather than simply remembering which operations had been performed in the past, TTSE goes further, enabling the Agent to accumulate both knowledge of the environment and execution methods that are effective under specific conditions.

The key to turning accumulated experience into improved capabilities lies in proper reuse.

The two office cases demonstrate the practical value of TTSE: the Agent does not have to explore from scratch every time, but can transform experience validated in previous tasks into a reference for subsequent executions.

This is precisely how RSI aims to achieve continuous improvement: every task execution should not only produce deliverables, but also create opportunities to accumulate effective experience for the next execution.TTSE’s dual-track mechanism of FACT and TIP makes this accumulation of experience more verifiable, updatable, and reusable.

However, do these benefits exist only in specific tasks, or can they also be realized across a broader range of office tasks?

TTSE Demonstrates Comprehensive Benefits in Office Benchmarks

To further evaluate TTSE’s performance in comprehensive office settings, the team tested the impact of integrating TTSE with WorkSwarm on the industry’s open-source leaderboard, WorkBuddy Office Bench.

It not only improved task performance but also effectively reduced token consumption and tool-call overhead: the evaluation results showed that, compared with the native implementation, task performance improved by 3.81%, input token consumption fell by 1.70%, output token consumption fell by 30.09%, and the average number of tool calls decreased by 2.

In Closing

We are at an inflection point at which Agents are moving from “one-off tools” to “continually evolving partners.”

The core value of RSI is not merely enabling Agents to complete more tasks, but enabling them to obtain feedback from every real-world execution, accumulate experience, and turn that experience into the capacity for continuous improvement.

From dual-dimensional optimization of Harness and Artifacts to TTSE’s dual-track evolution of environmental facts and execution techniques, openJiuwen is exploring different paths for enabling Agents to continuously self-improve: paths that optimize work methods and deliverables while continually refining their understanding of the environment and distilling reliable experience.

TTSE provides a verifiable, traceable implementation mechanism for experience-based self-evolution, whichenables Agents not only to learn “how to do it,” but also to understand “under what conditions this is effective,” and to continually improve themselves through repeated task executions.

Experience should not be fleeting. Helping an Agent learn both “how to do it” and “what the world is like” is the first lesson on its path toward long-term autonomy.

Academic Paper: TTSE: A Two-Track Online Self-Evolution Framework for LLM Agents
Original paper:
https://arxiv.org/pdf/2609.24289

The TTSE Dual-Track Evolution Rail is now available in WorkSwarm version 0.2.8. Interested users can visit the official website, download it with one click, enable experience-based self-evolution, and try it out immediately:

Try WorkSwarm now:
https://openjiuwen.com/workswarm
GitHub repository:
https://github.com/openJiuwen-ai/
AtomGit repository:
https://atomgit.com/openJiuwen/

If you are also working on Agent/LLM self-evolution, memory, and skill systems, feel free to join the discussion in the comments:

What do you think an Agent should “remember” most: a technique, or a rule about the environment?

*This article is published by QbitAI with authorization; the views expressed are solely those of the original author.