On November 10, Anthropic released its next-generation model, Fable 5.2, once again pushing the physical action capabilities of general-purpose large models to new heights.
According to the evaluation results published by Anthropic, Fable 5.2 achieved a 64.9% accuracy rate in the RoboDojo Sim dual-arm robot simulation evaluation—a substantial improvement over the previous generation, Fable 5.1—and also significantly outperformed GPT-6 Astra. Anthropic also demonstrated the model’s ability to complete robotic tasks such as twisting bottle caps, manipulating Rubik’s cubes, and drawing. Physical action by general-purpose large models is becoming a new focal point in competition among frontier models.
As of November 11, Claude Fable 5.2 ranked second on the Robodojo Sim Agent leaderboard, behind only Q0-Preview, which entered the leaderboard on October 30, with a task success rate of 68.87% and an overall score of 74.2. Q0-Preview was also the first system since the launch of the leaderboard to exceed an overall score of 70. Its technical approach is even more noteworthy than its scores. Rather than following the VLA or WAM technological paradigm, Q0-Preview adopted an agent-based approach, exploring how to leverage the combined capabilities of frontier multimodal large models—including task planning, spatial perception, memory feedback, and tool use—to autonomously accomplish complex physical manipulation tasks.
Surprisingly, the foundation model powering Q0-Preview was not the widely acclaimed GPT 6 Astra, but Claude’s previous-generation model, Opus 5.5! Yet Anthropic’s official report put Opus 5.5’s accuracy at just 39.4%. As general-purpose large models begin accelerating into the physical world, deeper questions consequently emerge: Where are the boundaries of their physical action capabilities? How can the spatial understanding and reasoning capabilities of large models be integrated with embodied know-how to achieve better physical action capabilities?
As a technical preview of Q0-Preview, this article briefly introduces Qianzhang Intelligence’s (Qoo Tech’s) exploration of Physical Agent.
The official evaluation code for Q0-Preview has been open-sourced.
GitHub URL: https://github.com/XPolicyLab/XPolicyLab/tree/main/policy/Q0_Preview
An open question: What is the essence of driving a large model to perform embodied manipulation?
The core insight of Q0-Preview is to determine the contact position and contact pose for each stage of manipulation.
Take the simple task of grasping an apple as an example. After recognizing the apple as the target object, the model must determine which part of the apple to grasp, from which direction the gripper should approach, and in what orientation it should grasp it. By converting these judgments into the robot arm’s target pose and then generating executable actions, the problem is solved.
TAG31@@Core approach: Spatial computation is all that is needed for embodied manipulation
TAG34@@General-purpose multimodal agents already possess excellent multimodal perception and programming capabilities. However, for embodied control, whether and how these two capabilities can be fully utilized has become a new open question.Returning to the question we just raised, control itself amounts to predicting contact information and contact pose. Since a pose can itself be determined by multiple points, a multimodal agent can therefore serve as a bridge: it perceives multiple pixel points and computationally calculates the spatial coordinates of the contact information, thereby enabling manipulation.
Therefore, Q0-Preview proposes fully utilizing the model’s spatial perception and programming capabilities. Specifically, it identifies target pixel locations through monocular or multi-view observation and performs spatial computation by combining camera intrinsic and extrinsic parameters with pixel-coordinate calculation tools, mapping the target contact pixels in the image into the world coordinate system.
More specifically, for the gripper, the model can represent the gripper’s target pose and contact information through three spatial reference points: the midpoint of two ordered points determines the reference contact position, the line connecting them determines the gripping direction, and the third point specifies the approach direction. Thus, the representation of position and pose, together with contact information, is unified into a computable spatial computation problem.
The multimodal agent determines the subtask for the current stage based on observations and instructions, and outputs target poses and gripper commands. Downstream motion-planning components such as cuRobo convert them into continuous actions. After execution, the multimodal agent checks the results based on new observations and adjusts the next step, forming an “observation — computation — execution — feedback” closed loop.
Harness: Enabling Large Models to Truly Go from “Can Compute” to “Can Act”
Centered on the core approach described above, Q0-Preview builds a Harness that standardizes the model’s interactions with observations, tools, and execution modules.
1. Clarifying Execution Boundaries and Tool Calls
Standardize the observations, robot state, and prior information available to the model; establish reasoning and tool-call budgets, as well as output formats for control commands. Tasks proceed in short groups of actions, with visual checkpoints set at necessary intermediate stages to determine, based on execution results, whether to continue or replan. Specify how the model invokes off-the-shelf or custom-written programs as tools for image cropping, local zooming, and other functions that support spatial computation.
2. Accumulating Generalizable Manipulation Experience through Trial and Error in Simulation
During development, trial and error in simulation is used to summarize generalizable operation rules for models to reference. For example, use the wrist camera of an idle arm to assist observation and reduce occlusion; account for both collision avoidance and subsequent visibility when planning paths; and adjust movement speeds according to stages such as grasping, transporting, and placing. These skills can be learned in simulation and transferred to real robots.
3. Supplementing Task Requirements and Clarifying Completion Conditions
For tasks with relatively vague descriptions, Q0-Preview supplements the static task rules, completion conditions, and necessary operational recommendations, reducing discrepancies between the model’s understanding and the evaluation requirements. During execution, it does not access any real-time evaluator rewards or environment-verdict signals.
4. Managing Context and Task Memory
TAG72@@Retain the initial information, recent observations, and compact task memory; record confirmed progress, the current state, and upcoming objectives; and control the overhead caused by context growth. Meanwhile, clearly defined interfaces isolate upstream decision-making from downstream execution, allowing multi-stage operations to remain continuously connected.Capability Showcase
Worth noting, Q0-Preview delivers remarkable performance across multiple dimensions, and on some tasks its execution is even smoother than expert teleoperation.1. Synchronized Dual-Arm Manipulation
During the same action phase, set separate targets for the left and right arms and simultaneously advance operations that can run in parallel.
2. Cooperative Dual-Arm Manipulation
Assign roles to the two arms around a common objective, and coordinate execution according to dependencies between actions such as support and handoff.
3. Precise Manipulation
Determine position and orientation through spatial computation after multi-view observation, and correct deviations using new observations obtained after approaching the target.Relocate the target, adjust the pose, or revise the manipulation plan based on changes in the object’s state and the results of action execution.
4. Adaptive Adjustment
5. Memory
Retain previously obtained information and completed steps, use them to inform decisions at later stages, and complete tasks that depend on historical information.
The true challenge of physical intelligence has only just begun
The Q0-Preview proposed by Qoo Tech demonstrates the potential of frontier foundation models in embodied manipulation. Foundation models are bringing a new technological inflection point to the embodied field: combining semantic understanding, spatial reasoning, and tool calling with the critical constraints of robotic manipulation offers a simple, direct route to significant progress.
However, deploying foundation models in real-world embodied environments and tasks still presents severe challenges in multiple areas.
First are inference efficiency and cost. Frontier large models may consume millions of tokens and several hours of inference time to complete an apparently simple embodied manipulation task. Such time and economic costs still fall well short of practical deployment requirements. Reducing redundant reasoning and tool calls while preserving manipulation capabilities and improving decision-making efficiency is a problem that embodied agents must solve.
The more fundamental question is:Where exactly does a large model’s embodied manipulation capability come from?
GPT 6 Astra sounded a thunderclap as frontier foundation models ventured into the embodied field. At the time, speculation was widespread that GPT 6 Astra’s training data included a great deal of embodied EGO data, bringing a leap in spatial understanding and embodied manipulation capabilities. However, Q0-Preview’s practice based on Opus 5.5 led us to think further: among factors such as embodied data, code generation, tool calling, and even 3D generation, which actually made the most significant contribution to the large model’s physical manipulation capabilities? How do the model’s own capabilities interact with the design of the Harness? A single leaderboard score cannot answer these questions. They must be clarified through systematic controlled experiments before clearer directions can be provided for future data development, training objectives, and system design. In addition, relatively long inference latency limits current embodied agents’ ability to adapt to dynamic environments. In the real world, objects and environments do not wait for the model to finish thinking, and changes between observation and execution may invalidate the original plan. How to respond promptly to dynamic changes, combine vision with force and tactile feedback, and cope with disturbances, recover from failures, and maintain manipulation reliability over long-term operation still requires continued exploration.
Despite these unresolved challenges, the zero-shot generalization demonstrated by foundation models has already opened a new technological path for embodied intelligence. Q0-Preview’s exploration showed us that combining general model capabilities with domain knowledge and constraints in embodied intelligence can continuously expand the boundaries of what robots can do. The possibilities opened up by foundation models are signaling the arrival of a new inflection point in embodied intelligence!
The team will soon release a technical report and a faster, cleaner version of the code, supplementing the technical details, presenting more real-robot experiments, and further discussing the capability boundaries and future directions of the current approach.敬请期待！
Oops, need only translation. I accidentally included Chinese? The final should English. Let's provide "Stay tuned! ". But check instruction "Keep placeholders exactly as they are." Fine.
