By incorporating spatial understanding into the continual learning process of foundation models, can robots better cope with changes in the real world?
As a robot reaches out to grasp a fruit or vegetable, someone moves the target away. An object that was already in place is thrown back onto the table. In another set of experiments, the tabletop continuously rises and lowers, and the gripper must keep up with changes in the target’s height. In both grasping and placing, when spatial relationships change, can previously learned skills still be used?
Figure 1 SLA real-robot manipulation video
The Hong Kong University of Science and Technology (Guangzhou) and OLA Dimensions jointly addressed this question by proposing SLA (Spatial-Language-Action Model).
First, the results: In the public RoboDojo-Real real-robot evaluation, SLA-Preview ranked first with an average score of 42.3 and a full-task success rate of 33.9%. Compared with π0.5, its success rate was 21.1 percentage points higher, or approximately 2.6 times as high.
Figure 2 RoboDojo-Real real-robot leaderboard screenshot, with SLA-Preview ranking first in average score
In the research team’s eight categories of real-robot tests, SLA achieved an average object-level success rate of 83.8%, 24.0 percentage points higher than π0.5. From the public leaderboard to tests involving dynamic grasping and changing tabletop heights, SLA demonstrated its ability to handle changes in real-world scenarios.
The core idea behind SLA is to develop spatial representations that support actions in tandem with the robot’s learning of those actions. Spatial supervision was extended from the post-training stage for specific tasks to large-scale continual pretraining and was retained during downstream adaptation.
01 After Ranking First on the Real-Robot Leaderboard: How Robots Respond to Change
RoboDojo-Real covers three robot embodiments—the ARX X5, Piper, and Piper X—and 18 tasks. SLA jointly used demonstrations from all three embodiments for post-training, then used the same model to complete the tests.
In the same public evaluation, π0.5 achieved an average score of 22.9 and a success rate of 12.8%, while SLA achieved 42.3 and 33.9%, respectively. The score reflects progress on part of a task, whereas the success rate measures whether the entire task was completed.
The leaderboard results provide a common reference. To see the effect of spatial capabilities directly, we can also examine experiments involving moving targets and changing tabletop heights.
In the fruit-and-vegetable grasping experiment, the researchers first completed post-training with 100 demonstrations from basic scenarios, then applied the same policy under three conditions: basic scenarios, positional perturbations, and dynamic interference. During testing, an object might be thrown back onto the table, and a target being grasped might be moved by someone.
For the positional-perturbation and dynamic-interference conditions, no additional corresponding perturbation demonstrations were collected for post-training.
Figure 3 Basic fruit-and-vegetable manipulation, positional perturbations, dynamic interference, and few-shot cross-embodiment adaptation.
Under positional perturbations, SLA achieved an object-level success rate of 82%, compared with 52% for π0.5. When the target moved continuously, the respective rates were 76% and 42%. Under tabletop height adjustments and watch-grasping conditions in which the target’s height continually changed, the respective rates were 78% and 44%.
The cross-embodiment experiment also had a clear data condition: in addition to the original 100 Piper demonstrations, 10 Franka demonstrations were added for joint post-training, resulting in an object-level success rate of 78% for Franka. This result tests the ability to adapt with a small amount of data from the target embodiment.
Each of the eight deployment conditions evaluated 50 object instances, with success requiring the complete manipulation sequence for the object to be completed. Together, they highlight a practical question: when observed spatial relationships change, can the policy adjust its actions accordingly?
02 Spatial Understanding: How It Enters Robot Foundation Models
Vision–language–action (VLA) models connect images, instructions, and actions, allowing them to leverage knowledge acquired through vision–language model pretraining. But for robots to perform manipulation tasks, they must also handle the relationships among distance, orientation, object occlusion, and gripper motion.
Recognizing a cup and moving the gripper accurately to the cup’s side require not exactly the same abilities. If the object changes position, the approach path may need to change; if the grasping angle changes, the wrist orientation must be adjusted accordingly.
A common approach is to introduce geometric information such as depth maps and point clouds during post-training. Yet real-world data do not always have consistent 3D annotations: depth may be missing, camera positions and calibration states vary, and some public demonstrations contain only RGB images.
If spatial learning can rely only on clean, complete geometric inputs, it is difficult to fully exploit the heterogeneous robotic experience already accumulated.
SLA chooses to align spatial information with the VLA’s own intermediate visual representations during training. It inherits π0.5’s pretrained architecture and weights, learns spatial representations through continued pretraining, and continues to use spatial supervision while adapting to specific tasks during post-training.
Figure 4 Overall SLA framework. It unifies heterogeneous robot data and performs spatial alignment during continued pretraining and post-training.
Specifically, a frozen 3D perception model, OmniVGGT, serves as the spatial teacher. It reads multi-view RGB images and extracts spatial features using whatever depth or camera calibration information is already available in the samples.
While learning to predict actions, the VLA also aligns its intermediate visual features with the teacher’s spatial features. Spatial supervision thus enters the representations used for action generation, enabling the model to acquire geometric information as it learns manipulation.
To cope with different scenes, textures, and camera viewpoints, SLA also adaptively aggregates visual features across multiple layers based on the input. In the RoboTwin ablation study, multi-layer adaptive fusion achieved a Clean2Random success rate of 72.3%, 5.0 percentage points higher than the best fixed single-layer approach.
Another design is state dropout during training: with a certain probability, robot-state inputs are removed from some samples, encouraging the policy to rely more heavily on the current image and reducing its dependence on fixed state patterns in demonstration trajectories.
At deployment, the spatial teacher, alignment projector, and layer-selection gating are all removed, so depth and camera calibration need not be introduced as additional policy inputs. The spatial alignment branch adds no computation to policy inference. Long-horizon RoboDojo tasks additionally use historical image memory; their leaderboard results reflect the performance of the complete system, including this mechanism.
03 Six Types of Robot Data: First Unify the Physical Meaning of Actions
For spatial supervision to operate at a larger scale, a fundamental question must first be answered: how can data from different robots be brought together for joint learning?
SLA uses six data sources—InternData-A1, AgiBotWorld-Beta, RoboCasa365-Pretrain, RoboCOIN, DROID, and MolmoAct—covering both real-world and simulated data and both single-arm and dual-arm robots. Before filtering and processing, these sources contain approximately 1.71 million trajectories and 14,330 hours of demonstrations in total.
Figure 5 Data sources for continued pretraining. The figure shows the scale before filtering and processing; task counts were tallied separately for each dataset and then summed.
Beyond data scale, control representations must also be consistent. Different manipulators have different numbers and arrangements of joints, and the same set of joint values does not carry the same physical meaning. Mixing them directly can make learning difficult.
The team uses end-effector poses to describe states and actions and standardizes robot coordinate conventions: for a single arm, the origin is the arm base; for a dual-arm robot, it is the midpoint between the two arm bases; and the coordinate axes consistently point forward, left, and up. This gives movement direction, rotation, and gripper actions comparable physical meanings.
Rotation representations are likewise processed. Directly subtracting Euler angles can produce outliers at angular wraparound boundaries. SLA instead computes relative rotations through SO(3) composition, represents them as rotation vectors, and applies channel-wise normalization.
For highly redundant simulation trajectories, the team downsamples by source and task and filters out inapplicable or anomalous data.The continued pretraining stage also did not use post-training datasets, in order to reduce the risk of leakage into downstream evaluations.
This stage used 48 NVIDIA H100 GPUs and took approximately 110 hours. Data processing and spatial supervision together formed the foundation for continual learning.
04 As the Data Continues to Grow, Can Spatial Capabilities Grow with It?
If spatial understanding is to become a foundation-model capability, it must also withstand data-scale experiments.
The researchers sampled the full mixed dataset by task, using continued-pretraining data at four levels—5%, 25%, 50%, and 100%—and separately controlled whether spatial alignment was used in continued pretraining and post-training before comparing performance under the RoboTwin Clean2Random setting.
When spatial alignment was used in both stages, the success rate rose with data scale from 49.2% to 60.5% and 67.4%, ultimately reaching 72.3%. At every data scale, this combination outperformed the other three configurations.
Figure 6 Continued-pretraining data-scale experiment. The approach using spatial alignment in both stages performed best at every data scale.
With the full dataset, using spatial alignment only during continued pretraining achieved a success rate of 63.2%; using it only during post-training achieved 57.1%; and using it in neither stage achieved 48.2%.
These comparisons support SLA’s technical claim: spatial representations can first be learned from broader robotic experience and then strengthened through downstream adaptation. Within the scope of this experiment, training objectives and data coverage jointly affected the final performance.
Other public benchmarks further tested different capabilities. Across the 50 tasks in RoboTwin 2.0, SLA was jointly post-trained using 50 clean demonstrations per task, achieving success rates of 86.0% on Clean2Clean and 72.3% on Clean2Random, for an average of 79.2%. The Clean2Random result was 26.3 percentage points higher than π0.5, demonstrating the benefits in randomized scenes.
Across the 24 kitchen tasks in RoboCasa-Panda, using 50 downstream demonstrations per task, SLA achieved an average success rate of 68.4%. In LIBERO-Plus, after post-training with standard LIBERO demonstrations, the model was tested under perturbations to the camera, language, lighting, background, and other factors, achieving a weighted success rate of 87.4%. These results correspond to different evaluation settings and cannot simply be combined into a single general success rate.
05 From Learning Actions to Adapting to Change
The SLA research by the Hong Kong University of Science and Technology (Guangzhou) and the Euler Intelligence team extended spatial supervision to the continued pretraining stage of foundation models and evaluated its effects through leaderboards, ablation studies, and real-robot manipulation tests. How spatial capabilities emerge and how they scale with robotic experience therefore became questions that could be tested one by one.The existing results still have limits on their applicability: there are clear differences across different tasks and embodiments, while long-horizon manipulation, open-ended tasks, and larger-scale real-world deployments require further validation. What SLA demonstrates is a path for continually developing spatial representations while building on existing VLA capabilities.
Papers and Projects
Paper: SLA: Scaling Spatial-Language-Action Model via Continual Pre-training
arXiv URL: https://arxiv.org/abs/2611.08373
Project homepage and live experiments: https://embodiedsla.github.io/
- TAG147@@Code repository: https://github.com/OpenHelix-Team/Spatial-Forcing
RoboDojo real-robot leaderboard: https://robodojo-benchmark.com/leaderboard?bench=realworld
