Skip to content
#Evaluations and benchmarks
  1. 芯东西78

    Ant Group's Lingbo open-sources LingBot-Touch tactile pretrained model, unifying signals from multiple sensor types

    Ant Group's Lingbo on November 10 open-sourced LingBot-Touch, a tactile pretrained foundation model for dexterous manipulation that can encode different sensing signals such as piezoresistive and vision-based tactile signals into unified tactile tokens for use by different dexterous hands and policy models.

    Why it matters: The original article provides comparative data on the cross-sensor tactile pretrained model across 26 real-robot tasks, which can be used to assess the actual impact of tactile representations on policy success rates.

  2. The Robot Report88

    NVIDIA launches GR00T 2 world action model and Isaac Lab 3.0 at CoRL

    NVIDIA launched the open world action model GR00T 2 and the open-source robot learning framework Isaac Lab 3.0 at CoRL. GR00T 2 was trained on 100,000 hours of NVIDIA open-source first-person data; with only one demonstration per task, it reached an average success rate of 71% across 50 tasks, rising to 95% with five demonstrations, and without task-specific demonstrations it completed at least once each of 40 of the 50 tasks.

    Why it matters: GR00T 2's few-shot success rates and Isaac Lab 3.0's training throughput gains provide the current baseline for embodied models and the simulation training stack.

  1. arXiv cs.RO80

    Paper analyzes agentic manipulation policies: AMP1 achieves 85.6% zero-shot success rate

    The paper proposes agentic manipulation policies (AMP), in which a generalist AI model directly controls a robotic arm and adjusts its actions as the task progresses. The authors' reference implementation, AMP1, runs unchanged across single-arm, bimanual, and mobile manipulation.

    Why it matters: The paper reports zero-shot success rates and real-world trial data for a generalist AI model directly controlling a robotic arm, which can be compared with specialist policies such as VLA and WAM.

  1. 机器之心 · 微信公众号78

    Northwestern, CMU and others propose WOVEN: cross-domain gains on 22 benchmarks with 2k samples, plus a training recipe

    A team from Northwestern, CMU, UNC Chapel Hill and Amazon, in the paper "WOVEN: Weaving Visual World Modeling into Multimodal LLMs," defines world modeling as estimating the state transition p(s′|s,a) and builds WOVEN, a training and evaluation base of 36K questions.

    Why it matters: The paper breaks world modeling into independently intervenable state-transition tasks and provides reusable rules for selecting training data, offering reference value for multimodal model training.

  1. Xbotics具身智能实验室 · 微信公众号78

    GE-Sim 2.0 and π0.5 closed-loop simulation test: Why does reward hacking occur in VLA training?

    The Xbotics Embodied Intelligence Lab built a closed-loop simulation pipeline using the AgiBot GE-Sim 2.0 world model and the OpenPI π0.5 policy. In testing, the world model generated about 4.33 frames per second, the pure-policy closed loop ran at about 3.20 frames per second, and after adding MiMo online scoring, the rate dropped to about 0.88 frames per second.

    Why it matters: The author uses a four-stage test to break down the frame-rate bottlenecks and causes of reward hacking in world-model closed-loop simulation, which can be transferred to the design of VLA evaluation workflows.

  1. arXiv cs.RO80

    SimuGuard: Auditing Contact Artifacts in Simulation Benchmarks Such as LIBERO and RoboTwin

    The authors propose SimuGuard, an auditing plugin for LIBERO, RoboTwin, RoboCasa, and RoboDojo that identifies failures caused by the simulator rather than the policy. On RoboTwin, the authors trace these failures to the physics engine's collision detection: when geometry is 1 to 2 centimeters apart, the engine reports a penetration of about 2 centimeters, and the solver ejects the object at several meters per second because the distance algorithm discards triangles below a fixed area threshold.

    Why it matters: The original traces failures in simulation benchmarks to a physics engine collision detection constant and provides a before-and-after success rate comparison, which can serve as a reference for teams doing simulation evaluation.

  2. arXiv cs.RO77

    From Teleoperation to Long-Horizon Dexterity: Evaluating Visuomotor Skills and Offline Proxies for Dexterous Manipulation

    The research team used a five-fingered robotic hand to grasp and insert RJ45 connectors, evaluating 281 imitation-learning policies spanning seven policy families in approximately 1,700 real-robot trials. Controlled ablations showed that observation and action representations had a greater impact than conventional optimizer and sampling choices. For point-cloud policies, region-of-interest cropping, representation in the robot-base frame, and normalization within the point encoder each improved success by 70 to 100 percentage points. For the vision-language-action policy, switching to absolute action prediction improved insertion success by up to 70 percentage points.

    Why it matters: Approximately 1,700 real-robot trials compared 281 imitation-learning policies, indicating that observation and action representations determine real-world success more than optimizer and sampling choices.

  3. HuggingFace Daily Papers76

    RoboColosseum: A distributed real-world robot policy evaluation framework across four embodiments

    RoboColosseum is a distributed evaluation framework that evaluates general-purpose robot policies over a network of locally operated robots in multiple locations, using unified policy serving and standardized protocols.

    Why it matters: This distributed evaluation framework used 5,699 paired trials to compare VLA policies with frontier agents, revealing inconsistent rankings across embodiments and a ranking reversal after fine-tuning.

  1. 机器之心 · 微信公众号84

    Qiyuan releases PrimeSim world-model simulator, tops EWMBench and WorldSimProbe

    The Qiyuan Embodied Research Center has released PrimeSim, a world-model simulator for robot manipulation tasks, built by adapting a pretrained video generation model through mid-training and post-training stages. It ranks first on EWMBench with an overall score of 6.09, and took first place in three of five WorldSimProbe evaluations: global trajectory coverage, action-source behavior preservation, and interaction dynamics.

    Why it matters: The original text provides the two-stage training recipe and results across multiple leaderboards, which readers can use to judge the feasibility of adapting video generation models into simulators.

  1. 机器之心 · 微信公众号82

    X-Era AI opensources 4D world action model E0.5, real-robot success rate surpasses π0.5

    X-Era AI has open-sourced its native 4D world action model E0.5, achieving overall average success rates of 85.0% and 88.3% on the AgileX Piper dual-arm robot and the Zerith H1 humanoid robot, respectively, 13.3 and 26.7 percentage points higher than π0.5.

    Why it matters: The original text provides real-robot success rate comparisons between E0.5 and π0.5 on dual-arm and humanoid platforms, which can be used to assess the manipulation generalization performance of the 3D geometric backbone approach.

  2. 钛媒体81

    Astra's Shockwave and the Embodied AI Industry's 50 Days

    In the 50 days since GPT-6 Astra was released, the embodied AI industry has conducted intensive evaluations and discussions around it, and general-purpose large models have begun directly taking over robot operation. Tests by SenseTime and several institutions concluded that Astra is basically capable of completing embodied tasks, but still has a clear gap from practical use in real environments, underperforming in millimeter-level contact, dynamic control, and response speed. The article also reviews industry uses such as using Astra directly for policies, reviewing VLA actions, building simulation environments, and data annotation, as well as chain reactions including cooling financing and major companies stepping up data collection.

    Why it matters: It reviews evaluation results, company reactions, and industrial chain changes in the embodied AI industry within 50 days of GPT-6 Astra's release, presenting the differentiated landscape after general-purpose large models entered robot operation.

  3. 机器之心 · 微信公众号78

    AgiBot releases embodied model GO-3 Preview, enabling full-pipeline scaled learning from pretraining to post-deployment

    AgiBot has released the embodied model GO-3 Preview, focused on full-pipeline scaled learning across pretraining, post-training, and post-deployment, and it ranked first in three evaluations: RoboDojo, RoboTwin, and RoboColiseum.

    Why it matters: AgiBot disclosed GO-3 Preview's three-stage learning pipeline of pretraining, post-training, and post-deployment, along with specific benefit data from mixed training using real-machine clusters and world models.

  4. AgiBot72

    AgiBot releases embodied foundation model GO-3 Preview, ranks first on three embodied intelligence benchmarks

    AgiBot has released the embodied foundation model GO-3 Preview, built on its self-developed WRAM (World Reasoning Action Model) architecture that unifies task understanding, future state prediction, and action generation, and it ranks first on three embodied intelligence benchmarks: RoboDojo, RoboTwin, and RoboColiseum. It is planned to be fully open-sourced by the end of 2026.

    Why it matters: The original article fully discloses GO-3 Preview's three-stage design of pretraining, post-training, and post-deployment learning, which can be used to understand the engineering path for scaling embodied models across their full lifecycle.

  1. 简智机器人 · X67

    GenRobot.AI's two open-source datasets enter Hugging Face robotics dataset trending top five

    GenRobot.AI announced that both of its open-source datasets, Gen-HumanWholeBody and Gen-HumanEgo, have entered the top five of the Hugging Face robotics dataset trending list. Gen-HumanWholeBody is a first-person whole-body manipulation dataset containing 600 hours of whole-body manipulation data, 16,976 real-world scenes, and 1,281 human skills, with whole-body MPJPE of about 3 cm; the authors say it targets whole-body motion learning, human-to-robot transfer, and embodied foundation model research.

    Why it matters: Two open-source datasets entering the top five of the Hugging Face robotics dataset trending list can serve as a reference for observing the level of attention to whole-body manipulation data.

  1. Xbotics具身智能实验室 · 微信公众号77

    How to Choose a Robot Simulator: Same Success, Tenfold Contact Force Gap

    The article interprets a simulator comparison report released by Manda Robotics in early October, running about ten sets of experiments with the same Franka 7-axis robotic arm and the same control program on four physics engines: PhysX, MuJoCo, MuJoCo Warp, and Genesis.

    Why it matters: The article uses controlled comparison experiments under uniform conditions to show that simulator divergence is concentrated in contact forces rather than success or failure, and provides transferable troubleshooting and verification methods.

  1. HuggingFace Daily Papers78

    Mechanistic Study of VLA Models: Not All Features Are Created Equal

    The study proposes an interpretability pipeline and finds that injecting task A's activations into the action expert causes pi0.5 to execute task A's motor trajectory in 99.6% of episodes (n=1,968), and that the injected program is bound to absolute workspace coordinates while ignoring the visible scene, offering a mechanistic explanation for the perturbation brittleness reported by benchmarks.

    Why it matters: The study uses intervention experiments to provide a mechanistic explanation for the brittleness of VLA policies, helping readers understand why models fail after slight scene changes.

  1. 机器之心 · 微信公众号76

    Tsinghua team builds HandsOn dexterous manipulation benchmark with 66 tasks to evaluate robot bimanual capabilities

    Ding Ning's team at Tsinghua University, together with researchers from Beijing University of Posts and Telecommunications, Peking University, and Nature Will, built HandsOn, a simulation platform and evaluation benchmark for cross-embodiment dexterous manipulation, containing 66 manually verified tasks and 18,000 human simulation teleoperation demonstrations.

    Why it matters: Readers can learn why dexterous manipulation evaluation should distinguish execution from transfer, and which specific failure points current multi-task policies have on bimanual collaboration tasks.

  1. The Humanoid Hub76

    Ropedia releases open-world model W0, trained on about 10,000 hours of first-person human video

    Ropedia released the open-world model W0, trained on about 10,000 hours of first-person human video. A single set of weights can predict the first-person view of humans, robotic arms, and humanoid robots (Unitree G1, Fourier GR-1), using a shared camera-plus-skeleton representation; adding a new robot requires only its URDF.

    Why it matters: The original article provides the training data scale, the cross-embodiment transfer method, and measured comparisons, so readers can judge the world model's actual impact on robot policy training workflows.

  2. Xbotics具身智能实验室 · 微信公众号76

    SenseTime launches SenseNova-RoboRSI and a 27B embodied model, achieving SOTA on 12 simulation benchmarks

    The SenseTime SenseNova team launched the SenseNova-RoboRSI unified execution framework and the 27B embodied model SenseNova-RoboRSI-27B-Preview.

    Why it matters: Through comparison data from the same model with different execution systems, readers can intuitively understand the magnitude of the execution framework's impact on embodied task performance and its cost.

  3. 自变量机器人 · 微信公众号74

    X Square Robot releases X2Real simulation benchmark, with 0.96 correlation between simulated and real-robot task success rates

    X Square Robot has released X2Real, an evolvable simulation evaluation benchmark. Its simulated evaluation and real-robot evaluation achieve a task success rate correlation of 0.96 and a progress score correlation of 0.99.

    Why it matters: It provides correlation data between simulated and real-robot evaluation along with a unified integration method, allowing readers to judge whether simulated evaluation can replace part of real-robot testing.