Skip to content
#Data and simulation
Oct 27Tue
  1. arXiv cs.RO58

    Can Differentiable Simulators Give Better Policy Gradients? ICLR 2026 Paper Proposes DDCG and IVW-H

    An ICLR 2026 paper re-examines the question of whether differentiable simulators can give better policy gradients, arguing that bias from discontinuous dynamics is not the main obstacle. The authors propose DDCG, a lightweight test that switches estimators in nonsmooth regions; it requires only one hyperparameter and remains reliable with small samples. They also propose IVW-H, a per-step inverse-variance implementation that stabilizes variance in differentiable robot control tasks without explicit discontinuity detection. The paper concludes that estimator switching can improve robustness in controlled experiments, while in real deployment variance control is often more critical.

  2. arXiv cs.RO58

    Vision-Language-Action Model Robot Learning Framework for Agricultural Manipulation

    A research team proposed a simulation-to-real robot learning framework for agricultural reaching and grasping, building a MuJoCo-based agricultural environment with diverse tomato plant structures and collecting expert demonstrations via joystick teleoperation. Two pretrained generalist policies, Octo and π0.5, were first adapted with simulated demonstrations, then fine-tuned with real demonstrations on a Franka Research 3 robotic arm. Results showed that π0.5 scored higher than Octo on task performance and degraded less under distribution shifts such as viewpoint and spatial offsets; the study also examined the effect of proprioceptive information on policy performance.

  3. arXiv cs.RO63

    CLARO: Open-vocabulary grasping in cluttered scenes with a VLM agent

    CLARO is a VLM agent that orchestrates skills such as open-vocabulary object detection, segmentation, 6-DoF grasping, and non-prehensile pushing to locate and retrieve a language-specified target in cluttered environments. It records past actions to support long-horizon execution and outputs grasp poses or 3D waypoints, enabling transfer to different robot arms without task-specific training. In a dynamic simulation environment, CLARO achieved an average success rate of 89.1%, a relative improvement of over 16% on the strongest competing method in highly cluttered scenes; on a real robot arm, its success rate was 88.8%.

  4. arXiv cs.RO71

    PlayPolicy: Learning Dexterous Manipulation from Supervised Play

    The authors propose the PlayPolicy framework, in which a robot first learns a play policy from minutes of human play data through behavior cloning, then deploys it autonomously to collect hours of interaction data to train a world model; at inference time, this policy proposes candidate action sequences for model-predictive control (MPC).

  5. arXiv cs.RO51

    Illumination-aware active perception for spacecraft inspection

    The authors propose an inspection planning framework that replaces geometric visibility checks with a learned camera information metric based on a classifier variational autoencoder (VAE), to address inspection failures caused by drastic orbital illumination changes. The method implements a guidance, navigation, and control architecture in a photorealistic physics simulator and tests multiple orbital illumination conditions and viewpoints: average brightness reaches 0.57, with 81.6% of observations above 0.25, compared with 0.02 (3.0%) for geometric planning with illumination bands and 0.04 (6.0%) for pure geometric planning.

  6. arXiv cs.RO60

    CapPro: Adapting VLA models to new manipulation tasks with a single human video

    Researchers propose the CapPro framework, which lets a vision-language-action (VLA) policy adapt to previously unseen manipulation tasks using only a single human demonstration video, without robot demonstrations or fine-tuning for the target task. The framework has two components: a phase-aware human video representation module, which learns task-relevant query representations through visual token selection and phase-caption supervision, and a video-conditioned adaptation module, which injects the compact video and query representations into the action-conditioning context of a pretrained VLA policy.

  7. arXiv cs.RO62

    OneDemo: Learning policies and generating data from a single demonstration

    OneDemo proposes a reinforcement learning framework that edits a single successful demonstration into trajectories suited to new task configurations; the learned editor serves both as a task policy and as a reusable demonstration generator. On 15 Meta-World tasks, it reaches a 92.7% success rate using an average of 53.7k environment interactions; on 9 DexMimicGen tasks, it generates demonstrations with an average success rate of 98.4%, and BC-RNN policies trained on the generated data reach an average success rate of 99.6% at the same data scale.

  8. arXiv cs.RO62

    CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping and LLM Agent for Immersive Collaborative VR Teleoperation

    CognitiveReality builds a semantically indexed Gaussian-TSDF map from a robot's RGB-D stream, shared by a VR operator and a tool-calling language agent. A single mapping program adapts to different platforms through configuration alone, accepting poses from robot SLAM, joint kinematics, motion capture, or visual tracking, and handles localization outages with shadow tracking and keyframe-anchored PnP, maintaining open-vocabulary instance identities and per-object quality at 2 Hz.

  9. arXiv cs.RO60

    DGT-Map: Directional Global Traversability Mapping with Multi-Task Learning for Heterogeneous Vehicles

    DGT-Map proposes a self-supervised framework that learns global, directional, and vehicle-conditioned traversability costmaps from RGB-D observations and locomotion signals. It uses a shared multi-task backbone to learn common terrain features across vehicles, while vehicle-specific prediction heads preserve platform-dependent responses. At inference, it outputs a heading-indexed costmap for use by direction-aware planners.

  11. arXiv cs.RO59

    EPCC: Faster-than-Demonstration Planning with Slow Expert Demonstrations and Fast Play

    The authors propose Expert-Play Contouring Control (EPCC), which combines slow expert demonstrations with fast, non-expert play to achieve execution speeds faster than the demonstrations. Expert demonstrations train a latent trajectory generator whose predictions are reparameterized into a time-independent contour of task progress, while play data trains a world model of fast-action outcomes. At deployment, the planner optimizes actions along this contour and penalizes deviation from it.

  12. arXiv cs.RO78

    X-WBC: A Cross-Embodiment Foundation Model for Humanoid Whole-Body Control

    X-WBC is a cross-embodiment humanoid whole-body control foundation framework that separates relatively shared human motion semantics from embodiment-specific physical execution. It uses human-centered command tokens to align full human motion, robot reference motion, and sparse VR observations; a causal Transformer learns reusable temporal structure from mixed multi-robot rollouts, and lightweight robot-specific modules map the shared representation to each robot's proprioception and action space.

    Why it matters: The paper proposes a cross-embodiment humanoid whole-body control framework that decouples shared human motion semantics from each embodiment's execution, and validates joint training on nine simulated embodiments and four real robots.

  13. 雷科技70

    Xingyuan Intelligence releases cross-embodiment data engine Chorus, claims about 40x training data expansion

    On October 27, Xingyuan Intelligence released Chorus Zhongxing, a cross-embodiment data engine for robot pretraining that can convert human manipulation videos captured by smart glasses, chest-worn cameras, wrist cameras, UMI devices, smart headbands, and other devices into pretraining data adapted to different robot embodiments.

  14. arXiv cs.RO77

    AgiBot Team Releases GE-Act 2.0 World-Action Model Technical Report

    The AgiBot Research Team released a technical report on GE-Act 2.0, a world-action model whose generative and action components are all trained from scratch on manipulation data. It consists of a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM), and is jointly trained with KASO.

    Why it matters: The original text presents the component design and data-scaling curves for pretraining a world-action model from scratch, allowing observation of changes in zero-shot cross-embodiment transfer.

  15. arXiv cs.RO71

    Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL

    This work proposes using sample-based model predictive control (SMPC) in simulation as an automated expert to generate large-scale offline datasets, then training an off-policy reinforcement learning agent with purely sparse task rewards, eliminating the need for manual dense reward design. Combined with a low-level dynamic stability controller, the high-level agent's learned policies can exceed the original optimal control teacher on task objectives. The authors completed sim-to-real deployment on an arm-equipped Spot quadruped and a G1 humanoid; the paper is the accepted version for CoRL 2026.

  16. arXiv cs.RO58

    SHRIMP: Iteratively Revising Robot Task Plans with Natural Language

    The SHRIMP system lets users automatically generate hierarchical robot primitive plans using natural language, and iteratively revise those plans through re-prompting and explicit correction. After each revision, users can validate the plan in simulation and, once satisfied, execute it on the physical robot. A user study with 35 participants on tabletop kitchen task planning showed that SHRIMP improved users' perceived control and robot transparency.

  17. arXiv cs.RO72

    DSMS: Contact-Implicit Multiple Shooting for Dynamic Motion Retargeting

    The study proposes DSMS, a contact-implicit, direct-simulation-based multiple shooting framework that converts kinematically feasible reference motions into dynamically feasible whole-body trajectories. It handles contact, friction, collision, and joint limits within a nonlinear program without presetting a contact schedule. Compared with existing retargeting methods, DSMS accelerates motion-imitation reinforcement learning training and yields policies with high success rates and low tracking error. The authors achieved zero-shot sim-to-real transfer on the Unitree G1, completing walking, jogging, contact-rich crawling, and a 180-degree jump-turn.

  18. arXiv cs.RO72

    SMAT: Jointly training human-exoskeleton co-adaptation policies in simulation and validating on real subjects

    The study proposes Staged Multi-Agent Training (SMAT), a method that jointly trains a musculoskeletal human agent and a bilateral hip exoskeleton control policy in physics simulation to learn an assistance policy that accounts for human-device co-adaptation. The policy was transferred to a physical hip exoskeleton without subject-specific retraining and tested in eight healthy adults; indirect calorimetry showed that active assistance reduced net metabolic rate by 19.7% relative to the passive device (p < 0.001), the hip mechanical positive-power ratio was 0.984, and effective assistance was maintained at walking speeds and terrains beyond the training conditions.

  20. arXiv cs.RO60

    MPPI-Based Trajectory Planning for Drifting Target Search and Capture with Unmanned Surface Vehicles

    This study proposes a spatiotemporal informative planning method based on model predictive path integral (MPPI) control for unmanned surface vehicles (ASVs) to search for and capture multiple drifting targets. The planner couples target drift prediction with exploration objectives through a cost design, optimizes continuous trajectories over long horizons, and directly generates kinematic commands; in the interception stage, pure pursuit guidance is used to achieve physical capture. In simulation, the method captured 18% more targets than the strongest baseline, and the approach was validated in real-boat trials.

  21. arXiv cs.RO61

    MRO-GWM: Action-Conditional and Object-Centric Gaussian Splatting World Model for Rigid Objects

    The authors propose MRO-GWM, an object-centric Gaussian world model for learning action-conditional dynamics of 3D rigid objects. The model represents arbitrary shapes and multi-object scenes with Gaussians in an object-centric frame, and uses a temporal-spatial Transformer to predict rigid body motion from past Gaussians and future actions; training is based on multi-view reconstruction, which requires handling partial observations caused by occlusion.

  22. arXiv cs.RO74

    Crazyflow: A JAX-based GPU-accelerated differentiable drone simulator

    The authors propose Crazyflow, a differentiable drone simulator built on JAX that combines high-fidelity dynamics with GPU acceleration, scaling to millions of environments or swarms of thousands of drones on a single CPU and GPU. Real-world experiments show that it supports analytical-gradient policy learning, achieving high-accuracy tracking without domain randomization, and sampling-based obstacle avoidance at over 500 million steps per second. The authors also demonstrate in-flight reinforcement learning: after tossing a physical drone into the air, training a recovery policy from scratch took only 0.37 seconds, and the drone was caught before landing.

  23. arXiv cs.RO58

    Haptic Sorter: A Unified Planning Framework for Online Shape Estimation and Real-Time Pose Inference

    The authors propose Haptic Sorter, a unified model-based geometric framework that integrates robotic haptic perception, modeling, and manipulation planning for manipulation when object shape and pose are unknown. The framework uses Bayesian optimization to guide haptic exploration and approximates geometric boundaries with superellipses, describes quasi-static robot-object interaction with an adaptive manipulation potential that encodes object geometry, and proposes an online ordinary differential equation based on model prediction and tactile feedback for real-time pose inference. The system is deployed on a 2D robotic sorting task, varying object geometries to validate the framework's robustness and generalizability in both simulation and a real-world multi-arm setup.

  24. arXiv cs.RO58

    LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation

    The paper proposes Future-Experience Conditioning (FEC), which uses a latent representation of a short future video as a condition for closed-loop policies, for exploration and policy adaptation in multi-step robot manipulation. Future clips are generated in three stages: an LLM reasoner performs task-ontology reasoning initialized from the current scene state, a robot-free digital twin rolls out the intended object motion, and a mask-free video diffusion model synthesizes a robot-consistent future clip.

  25. arXiv cs.RO67

    WorldArena 2.0 released: expanding the embodied world model benchmark across modality, functionality, and platform

    WorldArena 2.0 has been released, expanding embodied world model evaluation along three dimensions: modality, functionality, and platform. In modality, it extends from vision-only to visuotactile; in functionality, it extends beyond policy evaluation and planning to using world models as interactive reinforcement learning environments for policy optimization; in platform, it extends from simulation-only to multiple simulators and real robot embodiments. The benchmark evaluates perception quality, interaction utility, and cross-platform performance under a unified protocol, and the project page provides code and demos.

  26. arXiv cs.RO72

    RoboEvolve: A Planner-Simulator Co-Evolution Framework for Robotic Manipulation

    RoboEvolve couples a VLM planner and a VGM simulator into a mutually reinforcing co-evolutionary loop, bootstrapped from unlabeled seed images to alleviate manipulation data scarcity. Experiments show it raises the base planner by 30 absolute points and improves simulator success by 48% on average, surpassing fully supervised baselines with only 500 unlabeled seed images and reducing initial reinforcement learning data by 50 times.

  27. arXiv cs.RO60

    BEACON: Cross-Domain Co-Training of Generative Robot Policies via Best-Effort Adaptation

    BEACON proposes a theory-guided cross-domain co-training framework that trains diffusion-based visuomotor policies using abundant source demonstrations and limited target demonstrations, and formulates co-training as a discrepancy-aware importance reweighting problem. The method introduces scalable instance-level discrepancy proxies, stochastic alternating updates of policy and weights, and a multi-source extension, improving robustness and data efficiency over target-only, fixed-ratio co-training, and feature-alignment baselines in sim-to-sim, sim-to-real, and multi-source manipulation settings.

  28. arXiv cs.RO67

    AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation

    AffordSim is a scalable data generator and benchmark that integrates open-vocabulary 3D affordance prediction into simulation trajectory generation. Given a natural-language task description, it synthesizes scenes, generates affordance queries, and samples region-constrained grasps, then uses motion planning to filter executable candidates.

  29. arXiv cs.RO71

    RoboGene: A Multi-Arm Morphology-Diverse Task Generation Framework for VLA Pre-Training

    RoboGene is an agentic task generation framework for single-arm, dual-arm, and mobile robots that automatically produces diverse manipulation tasks before VLA pre-training. It combines diversity-driven sampling, agentic self-reflection to assess task quality, and human-verified feedback memory for execution-grounded validation and continual improvement. It collected 18,000 trajectories and introduces task feasibility and diversity metrics.

  30. arXiv cs.RO70

    FlowHOI: Flow-Matching-Based Semantics-Grounded Generation of Hand-Object Interaction Sequences for Dexterous Manipulation

    FlowHOI is a two-stage flow-matching framework that generates semantically grounded hand-object interaction sequences, including hand poses, object poses, and contact states, conditioned on first-person observations, language instructions, and 3D Gaussian splatting scene reconstructions.

  31. arXiv cs.RO71

    UniManip: An Open-World Robotic Manipulation Framework Based on Agentic Operational Graphs

    UniManip proposes a unified framework centered on the Agentic Operational Graph (AOG) for task orchestration, state tracking, execution verification, and closed-loop failure recovery in open-world robotic manipulation. The method grounds high-level semantic intent into executable primitives through a physically grounded scene graph, and combines a safety-aware local planner to generate collision-free trajectories. It achieves state-of-the-art performance on LIBERO-Pro, and in real-world experiments its task success rate is 22.5% and 25.0% higher than leading VLA models and hierarchical baselines, respectively.

  32. arXiv cs.RO58

    CRAFT-E: An Interpretable and Editable Neuro-Symbolic Framework for Affordance-Grounded Object Retrieval

    CRAFT-E is a modular neuro-symbolic framework that combines an LLM-derived verb-property-object knowledge graph with CLIP-based visual alignment and energy-based grasp reasoning for affordance-grounded object retrieval. The system provides interpretable grounding paths, incorporates grasp feasibility into affordance inference, and supports graph edits without retraining.

  33. arXiv cs.RO71

    Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models

    The study proposes an automatic framework for generating robotic planning and execution failure data, builds the GuardianFail-36k failure reasoning dataset based on RLBench and BridgeDataV2, and trains Guardian, a multi-view reasoning vision-language model. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and the newly introduced UR5-Fail. When combined with LLM-based manipulation policies, it improves task success rates in both simulation and real-world deployment. Code, data, and models are publicly available.

  34. arXiv cs.RO62

    IndustryNav: Evaluating Spatial Reasoning and Safety of Embodied Agents in Dynamic Industrial Navigation

    The authors propose IndustryNav, a simulation environment with 24 warehouse scenes for closed-loop navigation to specified destinations, and introduce safety metrics that record collisions and near-collision warnings. Experiments with 11 leading open- and closed-source VLLMs and 3 navigation baselines show that agents can reach their goals yet collide with moving obstacles, exposing a gap between task success and safe navigation.

  35. arXiv cs.RO53

    Study proposes virtual nonholonomic constraints for nonprehensile robotic catching

    The paper proposes using virtual nonholonomic constraints (VNHCs) to address the problem of nonprehensile robotic catching, designing and controlling the motion of the end-effector directly based on the object's state before it is caught, without the need to predict the object's trajectory and generate a time-varying robot trajectory. In the planar case, the method can catch the object at a desired height and minimize the relative normal velocity at the instant of contact; the same VNHC function depends only on the target catching height and can cover a range of initial conditions. After contact, model predictive control (MPC) is used to bring the object and end-effector to rest at a specified position. The approach has so far been validated in simulation.

  36. arXiv cs.RO62

    High-resolution tactile observation makes hidden racket-control mechanisms learnable

    The research team used an 864-node tactile-inertial glove to synchronously collect more than 14,000 strokes at 100 Hz from over 70 participants across five skill levels, and proposed a mechanics-based two-layer framework that infers the control driving the racket and stabilizing the grip from tactile history and reconstructs its spatiotemporal force realization.

  37. arXiv cs.RO72

    Video World Models Need Predictable Latents

    This paper proposes making the latent space of video world models balance predictability with pixel reconstruction accuracy, achieved through temporal objectives for representation learning paired with a past-conditioned decoder architecture. Under the same training budget, inference cost is reduced by up to 5x, training speed increases by more than 7x, and pixel and perceptual errors in out-of-distribution and long-horizon prediction drop by 15-24%. The authors argue that temporal predictability should be an important design principle for video world model latent spaces and provide a concrete method for incorporating it into autoencoder training.

  38. arXiv cs.RO49

    Whole-brain recordings in fruit flies and a neuromechanical digital twin: how intrinsic goals in hierarchical exploration capture future volitional action

    The study combines whole-brain recordings in fruit flies with a neuromechanical digital twin to test whether agents trained with intrinsic motivation algorithms can reproduce the fly's spontaneous exploratory behavior in a virtual tethered-ball experiment. Agents select actions from a discrete library of low-level motor primitives and are trained without external reward using 20 intrinsic motivation algorithms; among them, DIAYN came closest to the fly in behavior and neural prospective encoding, and reproduced the additive and subtractive rules of interhemispheric kinematic encoding.

  39. arXiv cs.RO26

    Quadrotor Trajectory Generation and Control: A Simulation Study Based on Differential Flatness and Minimum-Snap Trajectories

    This project explores controller design and trajectory generation for a quadrotor. It uses a differential flatness approach to generate minimum-snap trajectories based on waypoint navigation, and a simple PD controller to track the trajectories. Simulation results show that the method can generate smooth trajectories for the quadrotor's position and velocity.