Content types
Benchmarks and evaluations
Test results, comparisons, reproductions, leaderboard changes, and disputes about evaluation methods for existing models, robots, or systems. New methods or benchmark designs belong under Research papers; promotional demonstrations are not automatically evaluations.
42 selected · 26 in the last 30 days
深圳市机器人协会 · 微信公众号 ·
HKU MMLAB releases PhysicalRSI 1.0, tops RoboDojo and sharply cuts token costs
The University of Hong Kong's MMLAB jointly released PhysicalRSI 1.0, a self-evolving embodied intelligence system baseline, achieving a score of 36.27 and an average task success rate of 31.38% on RoboDojo, ranking first.
Shanghai AI Laboratory - Embodied Intelligence ·
Shanghai AI Lab launches decision model Intern·Mingjue, outperforming Jev and releasing 0.8B, 2B, and 4B weights
Shanghai AI Laboratory has launched the decision model Intern·Mingjue (Decision), which outperforms Jev and is available in 0.8B, 2B, and 4B parameter versions. In RTX 4090 tests, the 0.8B and 2B models had an average end-to-end latency of about 33 milliseconds and about 30 decisions per second, while the 4B model was 44.16 milliseconds and Jev was 109.70 milliseconds.
Dexmal 原力灵机 · 微信公众号 ·
Force Spirit DM0.5 Tops All Four RoboColiseum Leaderboards
Force Spirit announced that DM0.5 has topped all four RoboColiseum leaderboards (instruction following, spatial understanding, perturbation adaptation, and general manipulation), making it the only model on the leaderboards to rank first in every evaluation.
自变量机器人 · 微信公众号 ·
X Square Robot releases WALL-SS next-scale autoregressive world model
X Square Robot has released WALL-SS, a next-scale autoregressive world model that uses a coarse-to-fine, layer-by-layer refinement generation architecture, breaking away from the traditional diffusion-model approach of generating video in a single segment. In testing, its action-following score was 0.29, the only nonzero score among the compared models; the overlap between generated trajectories and action instructions rose from 0.251 to 0.539; streaming inference can reach 60 seconds of footage without collapse; and in closed-loop policy consistency tests across 6 tasks and 5 training stages, the success-rate agreement between virtual and real-world tasks reached 93%.
智源研究院 · 微信公众号 ·
BAAI releases FlagEval-Robo embodied intelligence evaluation system and dual leaderboards
BAAI released the FlagEval-Robo embodied intelligence evaluation system on August 21, launching the FlagEval-Robo-Sim and FlagEval-Robo-Real leaderboards and publishing evaluation results for 12 open-source models. All models underwent equal post-training on the evaluation dataset.
PKU EPIC Lab ·
PKU EPIC Lab releases HumanTracker humanoid motion tracking benchmark and HumanScore metric
PKU EPIC Lab proposed the HumanTracker benchmark for evaluating humanoid motion tracking, accepted to ECCV 2026. The benchmark contains approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels; the accompanying HumanScore metric is trained on 12K motion pairs containing 24K motions, aligns with human preferences, and can reveal contact and stability failures that kinematic metrics often miss, such as foot skating and mistimed landings.
正行创新 · X ·
Harness VLA open-sourced, achieves 82.4% on LIBERO-Pro
@ChaoYuTHU has open-sourced Harness VLA, which adds a Harness Layer to organize VLA models without modifying their weights. On LIBERO-Pro, Harness VLA scores 82.4%, compared with Pi_RLinf at 50%, Berkeley RATS at 43.8%, and NVIDIA Cap-X at 18.2%. Striding AI posted this tweet and sees it as a possible sign of a shift from end-to-end VLA to Harness VLA.
Sunday Robotics ·
Sunday Robotics previews ACT-2 model with 99.1% zero-shot success rate in clothes folding
Sunday Robotics previewed its robot model ACT-2, which achieved a 99.1% (±0.3%) zero-shot success rate in clothes folding across diverse unseen home environments with zero per-home adaptation, covering 9 clothing categories across 785 autonomous attempts, with an average folding quality of 4.72/5.
智源研究院 · 微信公众号 ·
BAAI releases multimodal representation world model Wujie·RoboBrain Orca
BAAI has released the multimodal representation world model Wujie·RoboBrain Orca. Its core idea is to shift from predicting the next word, next frame, or next action to predicting the next world state: it first learns a unified latent world representation, then adapts to downstream tasks through three lightweight readout modules for text, images, and actions.
X Square Robot/自变量机器人 ·
X Square Robot releases cross-modal embodied action tokenizer X-Tokenizer, with 13.5% gain in multimodal alignment and 8.25% gain in long-horizon tasks
X Square Robot has released X-Tokenizer, a cross-modal embodied action tokenizer built on a semantic residual quantization (SRQ) architecture and pretrained on 17 robotic arm series and 2.4 million trajectories (including 2 billion action frames). Compared with FAST, it improves multimodal alignment by 13.5% and long-horizon task performance by 8.25%, and scores 82.8 on RoboTwin 2.0; in real-robot tests it outperforms comparison methods such as FAST and 4-level RVQ.
HUMANOID ·
KinetIQ Ascend uses real-world reinforcement learning to boost humanoid robot manipulation throughput and success rate
HUMANOID has launched KinetIQ Ascend, an end-to-end visual reinforcement learning method that operates under production deployment conditions, extending its KinetIQ framework to let robots improve manipulation skills through trial and error rather than relying solely on human demonstration; the company says this is the first such method publicly demonstrated on production-line humanoid robot manipulation.
Robot.com Holdings Inc. ·
Robot.com releases R-KNav 10,000-hour sidewalk navigation dataset and launches FoMo Challenge
Robot.com has released the R-KNav dataset and launched the FoMo Challenge, inviting the community to train visual navigation models on real data from its R-kiwi sidewalk delivery robot fleet and to evaluate them in controlled tests on physical robots.
Berkeley AUTOLAB ·
CaP-X: Benchmarking and Improvement Framework for Robot Manipulation Coding Agents Released
Teams including Berkeley AUTOLAB propose CaP-X, an open framework for systematically studying Code-as-Policy agents in robot manipulation, centered on the interactive environment CaP-Gym and CaP-Bench, which evaluates 12 frontier models.
Dexterity ·
Dexterity details evaluation methods and production metrics for the Foresight packing agent
Dexterity published an article introducing system evaluation and production measurement for the Foresight packing agent. On the planning side, it was tested on four real warehouse box-type distributions, with density of 0.65–0.77 and stability of 99.7–100%, and distribution-free order statistics were used to give 95% confidence intervals; the article also found that randomly shuffling the box sequence causes local variance to collapse by about 1.4 times, making evaluation overly optimistic.
Toyota Research Institute Robotics ·
Toyota Research Institute explains statistical thinking for robot policy evaluation: from rigorous A/B testing to effective visualization
The Toyota Research Institute (TRI) published a tutorial article introducing a statistical A/B testing method based on success/failure labels for robot policy evaluation.
Physical Intelligence ·
Physical Intelligence proposes a physical intelligence layer and publishes π 0.6 deployment data from Weave and Ultra
In a blog post, Physical Intelligence argues that robots need a reusable physical intelligence layer, and it published real-world deployment data from partners Weave and Ultra using π 0.6.
Physical Intelligence ·
Physical Intelligence fine-tunes π 0.6 to take on five Robot Olympics tasks, winning three golds and two silvers
By fine-tuning its latest π 0.6 model, Physical Intelligence achieved preliminary solutions for the gold-medal tasks in 3 of the five everyday manipulation challenges in the Robot Olympics proposed by Benjie Holson, and reached silver-medal level in the remaining 2, with an average success rate of 52% and task progress of 72%.