Gamma Intelligence, TeleAI and Unitree have jointly launched LocoManiLab, a unified evaluation framework for humanoid robot mobile manipulation. It covers five types of home spaces, 13 scenario instances and 234 tasks, with the Unitree G1 and Dex3 dexterous hands as the unified test platform.
Flex-RSI is a benchmark for evaluating agents' recursive self-improvement capability under a fixed budget, covering perception, reasoning, robot action, 3D reconstruction, visual creation, and infrastructure tasks. In each round, an executor improves the task solution, which is kept only if it scores higher on held-out validation data; an improver then revises the harness, including the executor and itself. Final solutions are scored on hidden test data.
The paper proposes agentic manipulation policies (AMP), in which a generalist AI model directly controls a robotic arm and adjusts its actions as the task progresses. The authors' reference implementation, AMP1, runs unchanged across single-arm, bimanual, and mobile manipulation.
Why it matters: The paper reports zero-shot success rates and real-world trial data for a generalist AI model directly controlling a robotic arm, which can be compared with specialist policies such as VLA and WAM.
The authors propose the Seeing What Matters benchmark to evaluate task-conditioned actionable spatial belief in simulated tabletop manipulation, covering 860 frozen evaluation cases across three scenarios—coffee workstation, drawer/cabinet, and microwave—which expand to more than 115,000 evaluation episodes.
The research proposes having GPT-6 Astra read a robot's base policy weights and edit only 65 rows of token embeddings, raising the success rate on π0.5's 765 held-out paraphrased instructions from 86.3% to 92.0% (48 repairs, 4 regressions, paired p=1.31×10⁻¹⁰), all without gradient updates or new demonstrations.
The study proposes a planner-verified counterfactual (CF) test: holding the grasp pose fixed, a solid obstacle is inserted into the approach corridor and its points are added to the model input, after which the planner re-derives the label, flipping it from feasible to infeasible. Across eight predictors spanning point clouds, voxels, depth, and object tokens, CF-pair AUROC falls from 0.920-0.996 in natural scenes to near chance (0.470-0.504), and this flip holds across four planners, two robotic arms, and a real-object obstacle.
A team from Northwestern, CMU, UNC Chapel Hill and Amazon, in the paper "WOVEN: Weaving Visual World Modeling into Multimodal LLMs," defines world modeling as estimating the state transition p(s′|s,a) and builds WOVEN, a training and evaluation base of 36K questions.
Why it matters: The paper breaks world modeling into independently intervenable state-transition tasks and provides reusable rules for selecting training data, offering reference value for multimodal model training.
A team from Xiamen University and Nanyang Technological University proposed RoTA-Bench, which uses interactive 3D-to-2D projection alignment to evaluate whether vision-language models can precisely adjust an object's 3D pose based on visual feedback. The benchmark contains 674 tasks, of which 100 form a frozen test set; models must choose among 24 discrete rotation actions so that the projection reaches an IoU of 0.90 with the target contour.
Tristan, co-founder of YC F26-incubated company Asim, said the team is building a standardized benchmark for physical intelligence in humanoid robots, with a focus on evaluating dexterous manipulation capabilities. The benchmark uses repeatable scenarios to examine spatial reasoning, grasp adaptation, coordination, and how robots respond to physical interaction, with the goal of pinpointing where capabilities fail and providing reproducible failure cases. The team also noted the need to address comparability issues caused by differences in hardware, sensors, and control interfaces.
The authors propose VABAR, a Variational Automation Benchmark for agentic robotics, built on open-source tools and the Newton physics engine for simulated manipulation evaluation. The benchmark covers six manipulation task classes: tool sorting, tool hanging, gear meshing, syringe packing, cable insertion, and cable routing. It includes standardized task variants, difficulty levels, success metrics, 33 generalizable skills, and human-written programmatic reference solutions. The paper also presents an initial baseline evaluation based on Graph-as-Policy (GaP) and Isaac Lab Arena.
The authors propose Compounding Physical RSI, a self-evolving policy framework that uses automated capability-gap assessment to locate the task stage where a failure occurs, refines policies with behavior-regularized offline reinforcement learning, and builds candidates in weight space to preserve existing skills.
The study proposes a failure-boundary profile evaluation method in which each scenario family varies only one measurable factor (such as corridor width or the time gap in oncoming traffic), and each variant is labeled from measurement as one where the agent should proceed or one where it should stop. Along this axis, two curves are estimated for each agent: commitment (the probability of attempting the passage) and competence (the probability of passing without contact). The method locates where each curve crosses 0.5 and gives intervals that account for the simulator's run-to-run variance, while also using certified labels to derive a decision accuracy metric.
The study proposes RACE, a robot policy checkpoint scoring metric that requires no real-world rollout. It uses a fixed set of held-out successful trajectories to calibrate action deviations against state-conditional thresholds, then applies a running product to reduce the weight of later reference predictions after large deviations. In simulation, RACE outperformed baselines across different validation sources, while some baselines degraded noticeably when the validation set changed; on a real robot, its rankings were consistent with deployment success rates across the evaluated policy backbones and tasks of varying difficulty.
Google and Mercor released the Physical-Engineering's Last Exam (PLE) benchmark, which evaluates engineering agents on 85 mechanical, electrical, civil, and structural engineering tasks.
The study proposes Route Anything, a unified framework for studying model and policy routing in general-purpose AI systems, and establishes an optimal routing theory under quality-cost trade-offs. The authors construct an omni-modal trajectory matrix covering 25 benchmarks across language, image, video, speech, coding and workplace agents, and robotic manipulation, with 452 candidate configurations and 374,401 scoring runs.
Tsinghua University and UC Berkeley jointly proposed and open-sourced uni-robot-agent, a cross-environment unified embodied agent harness for connecting tool interfaces across different robots and simulation environments. Based on this framework, Qwen3.8-Flash-Next scored 31.39 on the RoboDojo benchmark, exceeding the publicly reported 28.97 for GPT-6 Astra, at about 1/20 of Astra's cost.
The Xbotics Embodied Intelligence Lab built a closed-loop simulation pipeline using the AgiBot GE-Sim 2.0 world model and the OpenPI π0.5 policy. In testing, the world model generated about 4.33 frames per second, the pure-policy closed loop ran at about 3.20 frames per second, and after adding MiMo online scoring, the rate dropped to about 0.88 frames per second.
Why it matters: The author uses a four-stage test to break down the frame-rate bottlenecks and causes of reward hacking in world-model closed-loop simulation, which can be transferred to the design of VLA evaluation workflows.
The authors propose SimuGuard, an auditing plugin for LIBERO, RoboTwin, RoboCasa, and RoboDojo that identifies failures caused by the simulator rather than the policy. On RoboTwin, the authors trace these failures to the physics engine's collision detection: when geometry is 1 to 2 centimeters apart, the engine reports a penetration of about 2 centimeters, and the solver ejects the object at several meters per second because the distance algorithm discards triangles below a fixed area threshold.
Why it matters: The original traces failures in simulation benchmarks to a physics engine collision detection constant and provides a before-and-after success rate comparison, which can serve as a reference for teams doing simulation evaluation.
The authors propose the MolmoSpaces-CommonSense benchmark, which builds a reasoning taxonomy grounded in developmental cognitive science and examines commonsense reasoning about occlusion, containment, support relations, semantically correct grasp points, spatial affordances, and hidden objects.
The research team used a five-fingered robotic hand to grasp and insert RJ45 connectors, evaluating 281 imitation-learning policies spanning seven policy families in approximately 1,700 real-robot trials. Controlled ablations showed that observation and action representations had a greater impact than conventional optimizer and sampling choices. For point-cloud policies, region-of-interest cropping, representation in the robot-base frame, and normalization within the point encoder each improved success by 70 to 100 percentage points. For the vision-language-action policy, switching to absolute action prediction improved insertion success by up to 70 percentage points.
Why it matters: Approximately 1,700 real-robot trials compared 281 imitation-learning policies, indicating that observation and action representations determine real-world success more than optimizer and sampling choices.
RoboColosseum is a distributed evaluation framework that evaluates general-purpose robot policies over a network of locally operated robots in multiple locations, using unified policy serving and standardized protocols.
Why it matters: This distributed evaluation framework used 5,699 paired trials to compare VLA policies with frontier agents, revealing inconsistent rankings across embodiments and a ranking reversal after fine-tuning.
Xinzhiyuan reported on the SpatialScout benchmark, which gives a camera to an AI and places it in previously unseen virtual rooms, where the model autonomously decides when to turn, move, and capture images, and submits the 3D coordinates of target objects. The formal evaluation selects 80 indoor scenes and 3,140 target instances from iTHOR, RoboTHOR, ArchitecTHOR, ReplicaCAD, and ProcTHOR, with at most 100 interaction rounds per task; scoring considers both mean IoU and the proportion of IoU above 0%, 50%, and 90%.
A research team proposes uni-robot-agent, a unified framework to support further training of open-weight robot-use agents. Without privileged inputs, the out-of-the-box Qwen3.8-Flash-Next achieves a RoboDojo score of 31.39, surpassing the published GPT-6 Astra-RoboProbe result of 28.97, and performs competitively on 19 real-world tasks.
A research team from Aether AI and UC San Diego has proposed RobotAgentBench (RAB), a comprehensive benchmark for open-world robot autonomy, building 50 household layouts, 90 atomic tasks, 18 short-horizon tasks, 8 long-horizon tasks, and 4 types of open-ended scenarios.
Princeton University, UT Austin, the University of Toronto, NVIDIA, and AgiBot jointly launched WorldOracle, an action-conditioned world model benchmark that uses the same initial frame and action sequence to compare model predictions with real execution.
Why it matters: The benchmark compares world model predictions frame by frame against real execution of the same actions and releases failure samples, making it easier to identify sources of model deviation over long sequences.
The You Yangwei team presents three tactile manipulation results: PACE was accepted as a CoRL 2026 Oral and achieved an average success rate of 90.7% across five real-world robotic insertion tasks; ForeTac uses future tactile prediction for action generation.
In an interview, Raphael Memmesheimer, head of the University of Bonn's NimbRo@Home team, said his team won the RoboCup 2026 best paper award for its paper "Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation."
The authors propose BiDex-100, a simulation benchmark for bimanual dexterous manipulation comprising 106 tasks organized along five capability dimensions: generalization, instruction following, limited demonstrations, long horizon, and memory. Each task is paired with an out-of-domain variant featuring controlled visual and spatial changes for evaluating robustness. The benchmark supports two embodiments and provides 8,080 success-validated teleoperated demonstrations totaling 6.25 million frames. The authors evaluate seven recent vision-language-action models under a unified protocol, establishing baselines for capability-specific performance, robustness, and data efficiency.
Zhengshu Zhang proposes in an arXiv paper that for diagnosis to improve behavior when a robot faces unfamiliar physical conditions, six ordered empirical conditions must hold: a meaningful reference, identifiability of the physical condition, use of the acquired evidence, decision value, selection value relative to a fixed alternative, and safe realization.
The Boreas Road Trip (Boreas-RT) dataset has been released, containing 9 real-world routes and 60 sequences totaling 643 km of driving data. Each route is traversed multiple times, enabling comparison of different traffic and weather conditions in identical environments.
The authors propose the Just Action Model (JAM) framework, which breaks down the differences between VLA and WAM into three separable choices: backbone initialization, the loss objective on future observations, and whether that loss mechanism is retained at inference. JAM unifies both into a structure of a pretrained visual backbone plus an action expert, trained with an action loss and a foresight loss.
The authors propose Constraint-Referenced Strategy (CRS), which projects the end-effector twist commanded by a policy onto the object's joint axis, separating progress along the joint direction from off-constraint waste. On 74 simulated articulated objects, binary success rate only shows a lower transfer failure rate without attribution, whereas the two CRS channels give each a distinct signature.
The paper proposes a denoised counterfactual protocol that uses bitwise-reproducible branching, repeated continuations, held-out oracles, and common random numbers to measure the selectable space a world action model (WAM) can achieve when selecting candidate actions at test time, and builds a fail-gated selector on that basis.
Researchers propose SO-101 Bench, a language-conditioned spatial manipulation simulation benchmark built on an Isaac Lab digital twin of a real SO-101 setup, comprising four task types, 138 objects, 916 evaluation episodes, and 3,629 simulated demonstrations.
EmbodiedLab is a benchmark that requires robots to "acquire knowledge through action," covering 30 quantitative science experiments across 5 disciplines on 14 apparatus platforms. The parameters of each instance are resampled, so results cannot be known in advance but can be scored precisely.
The authors propose JRDB-AVR, an active visual reasoning benchmark built on real JRDB robotics data: after receiving a visual reasoning question, an embodied agent must request limited observations by timestamp and viewing angle, and evaluation examines both the final answer and the visual evidence supporting it.
H2RBench is a Real2Sim benchmark for evaluating human-to-robot (H2R) transfer methods. It establishes a standardized protocol based on real human video demonstrations and simulated robot demonstrations, covering four categories of manipulation tasks.
WorldArena 2.0 has been released, expanding embodied world model evaluation along three dimensions: modality, functionality, and platform. In modality, it extends from vision-only to visuotactile; in functionality, it extends beyond policy evaluation and planning to using world models as interactive reinforcement learning environments for policy optimization; in platform, it extends from simulation-only to multiple simulators and real robot embodiments. The benchmark evaluates perception quality, interaction utility, and cross-platform performance under a unified protocol, and the project page provides code and demos.
The authors propose IndustryNav, a simulation environment with 24 warehouse scenes for closed-loop navigation to specified destinations, and introduce safety metrics that record collisions and near-collision warnings. Experiments with 11 leading open- and closed-source VLLMs and 3 navigation baselines show that agents can reach their goals yet collide with moving obstacles, exposing a gap between task success and safe navigation.
The study tested how text placed beside frames affects vision-language models judging robot videos: when a line saying "making progress" was added to clips where the robot was falling behind, Gemini, Qwen, and GPT judges flipped correct verdicts to "progress" 10 to 35 percentage points more often than with neutral text.