Changing the fundamental unit of RSI evolution from tasks to capabilities has the potential to move it from localized optimization for specific tasks toward continuous evolution across tasks and modalities.
Recently, Shenzhen Infinite Recursion Intelligence Technology Co., Ltd. proposed The Trinity, an omnimodal, all-task recursive self-improvement framework that uses capabilities as its fundamental unit of evolution and drives the co-evolution of environment generation and the Harness through collaboration among three roles: Creator, Explorer, and Evolver.
The framework first classifies tasks into a Task Tree, then analyzes the capabilities required for each task based on test trajectories, converts the Task Tree into a Capability Tree according to those capabilities, and finally evolves along the Capability Tree to obtain a Harness Tree.
After four rounds of RSI, The Trinity improved the average scores on 28 training benchmarks and 18 unseen benchmarks by 34.1 points and 26.2 points, respectively, and initially demonstrated the potential for transfer to real-world robotic tasks.
This work provides a new exploration path for Agents to move from task-driven localized optimization toward capability-driven continuous evolution across tasks and modalities.
Experimental Results
TAG19@@1. Multimodal Tasks: Four Rounds of RSI, with Unseen Tasks Continuing to Improve
On 28 cross-modal, cross-domain training Benchmarks such as ARC-AGI-3, buildarena, video mme v2, phyre, SOON, and CaP-X, The Trinity achieved an average improvement of 34 points, while achieving an average improvement of 26 points on 18 Unseen Benchmarks.
The first round brought the largest gains, but the following three rounds still yielded benefits, and performance on unseen tasks also increased with each iteration.
Of course, not every benchmark continued to rise. Some tasks reached a plateau early, and a few saw slight regressions. Overall, however, the system was able to accumulate improvements over multiple rounds on a fixed capability tree and transfer them to tasks outside the training set.
2. Embodied Transfer: Can What Is Learned in a Simulated Environment Be Applied to Real Robots?
Finally, we applied the evolved Harness to two real-world embodied tasks.
One was SOON: Unitree G1 needed to find the target table beside a fan in an office and walk to a position from which it could reach the table; the other was CaP-X: AlohaMini2 needed to inspect an upside-down cup, determine whether a cube was hidden inside, and report the cube's color and location.
Both tasks required the agent to continuously gather environmental information while acting and to make a decision once sufficient evidence was available. We applied the evolved Harness corresponding to the relevant nodes of the capability tree to these two tasks, prepared the task Skill through a reconstructed simulation environment, and then evaluated it on real robots.
Ultimately, the success rate of SOON on the real task increased from 0% to60%, while CaP-X likewise increased from 0% to 60%.
I. Bottlenecks and Challenges of Recursive Self-Improvement in Multimodal Agents
Giving an agent a task, having it execute the task, obtaining feedback, adjusting its strategy, and executing again—this is the basic idea behind much of the current work on Recursive Self-Improvement (RSI). Establishing such a feedback loop around several benchmarks and continually optimizing prompts, tools, and execution workflows by analyzing failed trajectories can indeed improve performance.
However, when we attempt to scale this approach to multimodal, long-horizon interactive tasks, the originally relatively simple feedback loop begins to face new challenges.
1. First comes the construction of environments at scale.From ARC-AGI-3 and interactive games to GUI operation, robot navigation and manipulation, the observation formats, state spaces, action rules, and success criteria vary from task to task. To keep agents continuously exposed to a broader range of tasks, new interactive environments must be continually constructed and validated. Clearly, the cost of relying on humans to do this work one by one cannot scale as the number of tasks keeps growing.
2. Harness iteration faces a similar problem.Agent performance depends not only on the model itself, but also increasingly on prompts, tool configurations, and execution workflows. As tasks become more complex and diverse, it is difficult for humans to design and optimize Harnesses fast enough to keep pace with changing task requirements.
A natural idea is:Since both manual environment construction and Harness optimization face bottlenecks, can these two processes also be handed over to agents to complete automatically?
We proposed an approach in which environments and agents co-evolve. This can indeed alleviate the scaling problem, but it raises an even deeper question.
Suppose a system can already automatically generate one hundred tasks and evolve a better Harness for each. It may ultimately have a hundred optimized solutions. Yet, are these one hundred improvements jointly advancing the system, or have they merely completed one hundred mutually independent local optimizations?
There are two interrelated questions here.
1. How experiences are organized. If experience always accumulates around individual tasks, the resulting optimizations tend to be limited to a particular environment and are difficult to transfer to other tasks. But if all experience is integrated into a single Harness, the requirements of different tasks may interfere with one another and constrain performance instead.
2. How improvements evolve. Even if every round produces a better policy, the system still needs a mechanism that preserves existing improvements, allows them to be inherited and reused in subsequent tasks, and gradually adapts and differentiates them as different capability requirements emerge, rather than optimizing everything from scratch each time.
It follows that an increase in the number of tasks does not mean improvements can continue to accumulate, nor do local performance gains necessarily produce cross-task capability growth.
What truly deserves consideration is:Can we find an organizational approach that transcends specific tasks, links experience gained across different tasks, turns each improvement into the foundation for subsequent improvements, and allows continual evolution through sustained iteration?
This is also the starting point for our reconsideration of what drives RSI.
II. Two Mechanisms: Co-Evolution and Shared Capabilities
We prefer to view the development of multimodal intelligence as a human-guided co-evolution between the Agent and the World.
The World provides tasks and feedback, while the Agent learns how to solve problems through interaction. In the past, humans performed most of this loop: selecting tasks, building environments, and then modifying the agent based on failure trajectories.
For this process to continue, at least two mechanisms are needed.
Mechanism 1: Agent–Environment Co-evolution, where the environment and the agent propel each other
Through co-evolution between the agent and the environment, the environment generates and adjusts training challenges based on the agent's performance, while the agent continually optimizes the Harness using feedback from the environment. By driving each other, they reduce reliance on manual environment construction and Harness iteration, making scaled recursive improvement possible.
Mechanism 2: Shared Capability Requirements, where different tasks may require the same underlying capabilities
Tasks may differ radically in modality, format, and objective, yet depend on the same underlying capabilities. For example, although ARC-AGI-3 and Micromouse have different task formats, both rely on accurate perception and state tracking.
This means thata single task may require multiple capabilities, while a single capability may serve multiple different tasks.If we organize experience around these shared needs, task feedback that was originally scattered has the potential to become learning signals that can be shared and transferred.
If we can identify common capability requirements across different tasks, we no longer have to treat each task as a completely independent learning objective. We can accumulate experience around these commonalities, then combine and specialize it according to the requirements of different tasks.
This is also the core premise behind Capability-driven RSI.
3. The Trinity: From Task-driven to Capability-driven RSI
TAG125@@Based on the preceding observations, we introduce The Trinity, placing environment generation and Harness evolution within the same RSI framework.In the system, a single model comprises three roles: Creator is responsible for constructing the world, Explorer for exploring it, and Evolver for analyzing feedback and guiding improvements to the other two. All three roles use fixed foundation models, and improvements occur primarily at the Harness level.
But the most important change here is not simply combining the three roles, butshifting the system's unit of learning from Task to Capability.
TAG1351. Task Tree → Capability Tree: Reorganizing ExperienceTAG139@@First, we collect multimodal tasks of different types, construct a Task Tree, and have the initial Harness execute these tasks to obtain verifiable interaction trajectories.Based on the task definitions and actual execution trajectories, the Evolver analyzes which capabilities a task requires and at which decision points failures occur.
TAG143@@We then no longer organize this experience according to the original task categories; instead, we reconstruct a Capability Tree based on shared capability requirements.The same capability can be shared across different tasks, while a task can require multiple capabilities simultaneously. Therefore, the paths in the tree represent combinations of capability requirements rather than hierarchical relationships among task categories.
The significance of this step is that experience originally scattered across different benchmarks now has a shared organizing basis.
TAG150@@2. Capability-Driven Real2Sim2Real: Generating Training Environments around Capabilities
TAG153@@With a Capability Tree, how should environments be generated?Traditional Real2Sim2Real often requires building a corresponding simulated environment for a specific real-world task. But if our goal is not to replicate the task, but to learn the common capabilities behind it, this opens up another possibility for environment design.
For a node in the Capability Tree, the Creator extracts the decision mechanisms that the related tasks jointly rely on and generates a simpler, verifiable simulated world.
For example, if several tasks all require verifying whether the evidence is sufficient before taking action, the training environment can be designed around this decision process without fully reproducing every visual detail, scene structure, and action rule of the original tasks.
TAG161@@The Explorer learns in these simulated environments, while the original benchmark is used to test whether the learned capabilities can be transferred back.The key here is not to make the simulated environment as complex as possible, butto preserve the decision mechanisms that need to be learned as much as possible while reducing irrelevant sources of complexity, thereby obtaining cleaner evolutionary signals.
TAG167TAG169@@3.Harness Tree: Making existing improvements the starting point for the next round of improvement
A Harness can be decomposed into different protocol modules, such as Perception, Memory, Inference, Planning, Orchestration, and Loop. Improving different capabilities may require coordination among different modules.
In The Trinity, the Explorer evolves the Harness node by node along the Capability Tree. At each node, it can either selectively inherit the modules already learned by the parent node or modify or add modules based on that node’s failure trajectories. Therefore, when a task requires additional capabilities, the system does not need to rebuild a Harness from scratch; it can continue improving upon the existing configuration.
Nor do the different branches need to be forcibly merged into a single globally optimal solution. They can share existing modules while retaining their respective modifications tailored to specific capability requirements.
After multiple rounds of iteration, these modules gradually form a Harness Tree. The Creator also continually improves its own Harness based on environment-validation failures and, at the end of each round, integrates validated module updates for use in the next round of environment construction.
4. Capability-Aware Router: Selecting a Different Harness for Different Tasks
The Harness Tree also raises a natural question: since different branches have already produced different specialized configurations, which one should be used when a new task is encountered?
We further introduce a Capability-Aware Harness Router. Based on publicly available information about the task, the capability descriptions of candidate Harnesses, and previously validated execution experience, the Router selects the Harness best suited to the current task. It does not need to know the task’s true capability labels in advance, nor does it need to include every module in a single Harness.
Only then is the logic of the entire method complete: tasks provide capability requirements, the Creator generates training environments around those requirements, the Explorer evolves modules based on feedback, the Harness Tree preserves existing improvements, and the Router applies the appropriate configuration to new tasks.
TAG191@@4. Experimental Analysis
TAG195@@1. Task vs. Capability: Does the Organizational Approach Really Affect What Is Ultimately Learned?
TAG199We compared different ways of organizing experience, as well as the gains brought by Capability-Driven Real2Sim2Real.
Although the initial construction cost of Sim is higher, the environments can be reused in subsequent iterations, and inference costs are also lower. This indicates that, compared with optimizing solely around tasks,organizing experience by capability and generating training environments for shared capabilities can yield better cross-task transfer, demonstrating that less is more.
TAG207@@2. Agent as a Router: After Specialization, Can the Right Harness Be Selected?
TAG210The Harness Tree allows different branches to optimize independently, but such specialization loses its value if the wrong configuration is selected for use. We therefore further compared different methods of Harness selection. This also explains why we do not want the final result to be merely one unified Harness that keeps growing larger: different tasks can rely on different combinations of modules, and the selection itself can also become part of what the agent needs to learn.
TAG214@@3.Capability–Module Relationships: Can Capability Decomposition Really Guide Module Design?
We first analyzed the relationships between 29 Modules and six categories of Capability after the fourth round of RSI. We found that different capabilities do indeed correspond to different module preferences. Capabilities can not only be used to organize tasks, but can also provide a basis for module selection and composition within the Harness.
4. Custom Capability Tree: From a Fixed Topology to Structural Exploration
The preceding experiments validated the effectiveness of Capability-driven RSI, but one question worthy of further exploration remains: how should the Capability Tree itself be constructed?
Currently, The Trinity constructs the Capability Tree based on the capability requirements of training tasks and performs Harness evolution under a fixed topology. This design allows us to observe more clearly the benefits of capability sharing, selective inheritance, and local optimization, but it also has certain limitations: once the tree structure has been determined, it does not dynamically adjust as learning progresses. Whether different capability decompositions, node combinations, and inheritance relationships affect the final improvement remains to be studied.
To this end, the team further open-sourced an exploratory experiment that supports custom Capability Trees. It allows researchers to define capability nodes, adjust the tree’s hierarchy and branch structure, and design corresponding simulation-based training environments for different nodes, enabling RSI experiments under different forms of capability organization.
This enables us to further explore several interesting questions: Which capabilities are suitable for sharing, and which should evolve independently? How do different tree structures affect Harness inheritance and specialization? As task types continue to expand, should the capability tree remain stable, or should nodes be allowed to continually split, merge, and reorganize?
These questions do not yet have definitive answers, but the custom experiments provide a starting point for exploring them.
If the current The Trinity demonstrates the feasibility of recursive improvement along a capability tree, then the next step worthy of exploration is to make the capability tree itself an object that can be optimized and evolved.Moving from continual improvement on a fixed capability structure toward the co-evolution of the capability structure and the Harness may also represent a direction for the further development of Capability-driven RSI.
Conclusion: What Drives Evolution, and What Does Evolution Shape?
Returning to the original question. If RSI always focuses on independent tasks, the system can easily become confined to their respective local objectives. Increasing the number of tasks may allow more solutions to accumulate, but this does not mean that the resulting experiences can be connected to one another, much less that they will naturally be distilled into reusable capabilities.
Expanding the scope of evolution, optimizing version selection and the seed task set, or allowing the decision-making Agent itself to participate in evolution are all attempts to improve the search and update efficiency of RSI.
Even if the system possesses sufficiently strong improvement capabilities and can record and retain all historical experience, a more fundamental question remains:
Which experiences are worth sharing? Which improvements need to be retained independently? How should the evolution of different tasks be connected to one another?
Capability-driven RSI attempts to answer these questions from a different perspective:Instead of treating the tasks themselves as the unit of optimization, it makes the shared capability requirements underlying those tasks the basis for driving improvement.
Following this approach, The Trinity organizes environment construction, experience accumulation, and Harness evolution under a common capability structure. On the one hand, it connects different tasks through shared capability requirements, enabling experience to be reused across tasks. On the other hand, it preserves distinct optimization directions for different capabilities through branched Harness evolution, allowing existing improvements to be inherited, adjusted, and combined.This organizational structure not only helps mitigate the risks of overfitting and reward hacking arising from task-level optimization, but also allows local performance improvements to gradually translate into the accumulation of more generalizable capabilities.
Paper and Author Information
Author Introduction:
The project lead for this work is Weidong Tang, an undergraduate at Xidian University whose main research interests include multimodal large language models, recursive self-evolution, and embodied intelligence. Ruiheng Wang, an undergraduate at Tongji University; Jingxuan Zhang, a master’s student at the National University of Singapore; and Hexin Wang, CEO of InfRec, were the main contributors. The corresponding authors are Pengfei Zhou, a doctoral student at the National University of Singapore; Haotian Zhang, a doctoral student at the Chinese University of Hong Kong; and Professor You Yang, the National University of Singapore President’s Young Professor.
Paper Title: The Trinity: Capability-Driven Recursive Self-Improvement for Multimodal Agents
Code: https://github.com/InfRec-Cardinal/The-Trinity
Paper: https://arxiv.org/abs/2610.36077
Project Homepage: https://trinity-rsi.github.io/
Hugging Face Daily Paper No. 1: https://huggingface.co/papers/2610.36077
Authors: Weidong Tang, Ruiheng Wang, Jingxuan Zhang, Hexin Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang, Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao, Kaipeng Zhang, Pengfei Zhou, Odin Zhang, Yang You
Affiliations: InfRec, XDU, TJU, NUS, UC Berkeley, CUHK, HKUST, HKU, SII.
Author contact: wdtang0705@gmail.com
