Skip to content
#Robot models and learning
Oct 28Wed
  1. arXiv cs.RO60

    Semantic World Models: Using Vision-Language Models to Predict Semantic Information for Robot Planning

    The paper proposes Semantic World Models, which model the world model as visual question answering about the semantic information of future frames rather than reconstructing future pixels, so that vision-language models can be obtained through supervised fine-tuning on image-action-text data. The authors validated it on simulated robot tasks, where it outperformed typical reconstruction-based action-conditioned world models.

  2. arXiv cs.RO60

    QUAN: Quaternion Approximate Networks Improve Image Classification and Oriented Object Detection

    The paper proposes Quaternion Approximate Networks (QUAN), which use Hamilton product decomposition to approximate quaternion convolution with real-valued operations, paired with custom CUDA kernels to achieve rotation-equivariant image classification and object detection. The authors also propose Independent Quaternion Batch Normalization (IQBN) to stabilize training and extend quaternion operations to spatial attention mechanisms.

  3. arXiv cs.RO72

    BiDex-100: A Bimanual Dexterous Manipulation Benchmark Covering 100+ Simulation Tasks

    The authors propose BiDex-100, a simulation benchmark for bimanual dexterous manipulation comprising 106 tasks organized along five capability dimensions: generalization, instruction following, limited demonstrations, long horizon, and memory. Each task is paired with an out-of-domain variant featuring controlled visual and spatial changes for evaluating robustness. The benchmark supports two embodiments and provides 8,080 success-validated teleoperated demonstrations totaling 6.25 million frames. The authors evaluate seven recent vision-language-action models under a unified protocol, establishing baselines for capability-specific performance, robustness, and data efficiency.

  5. arXiv cs.RO69

    IDTD: Information-Decoupled Trajectory Design for Sim-to-Real System Identification

    The authors propose an information-decoupled trajectory design framework, IDTD, which constructs an exploration objective from a Schur complement score derived from the Fisher information matrix, then normalizes by parameter, selects the optimal trajectory segment for each parameter, and aggregates the scores logarithmically, to mitigate unreliable identification caused by indistinguishable parameters.

  6. arXiv cs.RO52

    When Is Test-Time Physical Diagnosis Effective? Six Ordered Empirical Conditions and a Failure Analysis of Frozen Policies

    Zhengshu Zhang proposes in an arXiv paper that for diagnosis to improve behavior when a robot faces unfamiliar physical conditions, six ordered empirical conditions must hold: a meaningful reference, identifiability of the physical condition, use of the acquired evidence, decision value, selection value relative to a fixed alternative, and safe realization.

  7. arXiv cs.RO50

    Constraining World-Model Planning with Legal Reasoning: Validation of a Defeasible Deontic Logic Planning Stack on a Simulated Robotic Arm

    The paper proposes a legal planning stack that uses Defeasible Deontic Logic (DDL) to constrain a motion planner, leveraging a learned world model for planning and to supply legal context, thereby enabling ex ante intervention before an illegal action is executed. The stack was deployed in a scenario where a simulated robotic arm pushes blocks on a 3x3 grid. Results show that the legally constrained agent complied with norms at a markedly higher rate than the unconstrained one, and that modeling perceptual uncertainty further raised the compliance rate; legal reasoning ran efficiently at runtime, its rulings were auditable, and the stack adapted to exogenous signals and endogenous rule changes.

  8. arXiv cs.RO65

    Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control

    The authors propose a discrete-time neural reachability framework that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation, and uses gradient-free value probes to construct approximate teacher actions, turning adversarial actor learning into supervised policy learning.

  9. arXiv cs.RO66

    Learning-Based Informative Path Planning for Multi-Robot Systems: A Spatio-Temporal Gaussian Process Kalman Filter Framework

    The authors propose a grid-based spatio-temporal Gaussian process-Kalman filter framework for learning-based multi-robot informative path planning, representing anonymous target presence as a single latent variable field over a discrete workspace grid rather than maintaining one Gaussian process per target.

  10. arXiv cs.RO60

    Temporal Tactile Encoding and Compliant Control for Intent-Aware Robot-to-Human Bimanual Handover

    This study combines a VLA model with a compliant controller, fine-tuning the model using RGB observations, temporally encoded tactile feedback, and proprioception from human demonstrations to achieve robot-to-human bimanual handover. In a human-subject study, the system was compared against two baselines: one without tactile feedback and one with tactile feedback but no compliant control. Results show that compliant control and tactile history are complementary, with performance clearly outperforming the baselines. The project page is publicly available.

  11. arXiv cs.RO70

    From Grasping to Dexterous Manipulation: Large-Scale Grasp Pretraining for Dexterous Manipulation

    The research team proposes pretraining the low-level controller of a hierarchical imitation learning framework with large-scale dexterous grasp data, then fine-tuning it on downstream task demonstrations to support functional dexterity in articulated tool use. They built a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and introduced DexCraft, a simulation benchmark with six articulated tool-use tasks. In simulation and real-world experiments, the method outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch, improving real-world full-task success by 33.3 percentage points over DP3.

  12. arXiv cs.RO61

    LOPAL: Local Performance-Aware Active Learning from Imperfect Demonstrations

    LOPAL is a local performance-aware active learning method that uses a Gaussian Mixture Model (GMM) to jointly encode demonstration trajectories and their local quality assessments, thereby generating trajectories that outperform the imperfect demonstrations. The method also uses a shared autonomy (SA) mechanism to actively request user corrections in areas with insufficient data, while the robot autonomously executes its learned behavior during this process.

  13. arXiv cs.RO78

    FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors

    The research team proposes FTP-1, a generalist foundation tactile policy pretrained across multiple tactile sensors and embodiments. It uses heterogeneous encoders to map image-, array-, and state-based tactile signals into unified morphology-aware latent tokens, which are jointly modeled by a shared tactile Transformer expert.

    Why it matters: This work unifies multiple tactile signals into a shared policy and reports changes in success rates for cross-sensor transfer, offering a reference for tactile manipulation research.

  14. arXiv cs.RO70

    FAWAM: Force-Aware World Action Model for Closed-Loop Contact-Rich Manipulation

    The authors propose FAWAM, a force-aware world action model that incorporates force information at three levels: perception, prediction, and closed-loop execution. It first encodes historical 6-axis force/torque signals to modulate action generation, then jointly predicts future actions and end-effector wrenches to explicitly model contact evolution, and uses the predicted wrench trajectory as an execution-time reference while correcting actions online with real-time force feedback. In real-world experiments across multiple contact-rich tasks, FAWAM achieved an average success rate 36.25% higher than vision-only baselines and 21.25% higher than existing force-aware baselines.

  15. arXiv cs.RO64

    GTP-FA: A Two-Stage Grasp-Then-Plan Manipulation Framework with Failure Attribution

    The authors propose GTP-FA, a two-stage "grasp-then-plan" framework for robotic manipulation: it first generates grasp candidates, then performs downstream motion planning conditioned on the selected grasp. The framework learns a failure attribution model that outputs a distribution over failure modes for failed trajectories, and uses it to separately optimize the grasp scoring and planning modules. Simulation and real-robot experiments show that GTP-FA improves overall task success rates over baselines including RL, IL, diffusion policies, and VLA, and the paper has been accepted to NeurIPS 2026.

  16. arXiv cs.RO71

    RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models

    RePO-VLA proposes a recovery-driven policy optimization framework that assigns distinct roles to success, recovery, and failure trajectories for long-horizon, contact-rich manipulation tasks. The method first uses recovery-aware initialization to slice corrective segments into new episodes, then learns a progress-aware semantic value function, and uses value-conditioned refinement to make the policy prefer high-progress actions.

  18. arXiv cs.RO62

    RAPID: A Robotic Agent Platform for EV Battery Pack Disassembly

    The authors propose RAPID, a robotic agent disassembly platform that integrates a gantry-mounted industrial robotic arm, RGB-D perception, and an automated nut-running tool to remove fasteners on a full-scale EV battery pack. Open-vocabulary object detection achieves 0.9757 mAP50; among three one-shot fastener removal strategies, taught-in poses achieved a 97% success rate in 24 minutes, one-shot vision execution achieved 57% in 29 minutes, and visual servoing achieved 83% in 36 minutes (n=204).

  19. arXiv cs.RO65

    MO-Playground: Massively Parallel Multi-Objective Reinforcement Learning for Robotics

    The authors propose MORLAX, a GPU-native multi-objective reinforcement learning algorithm, and MO-Playground, a pip-installable GPU-accelerated multi-objective environment library. They can approximate Pareto sets within minutes, achieving 26-271x speedups over traditional CPU methods and up to 19x speedups over prior GPU methods, while learning Pareto front hypervolumes that are superior. The team also implemented a custom BRUCE humanoid robot environment, learning Pareto-optimal locomotion policies in simulation across 6 practical objectives such as smoothness, energy efficiency, and arm swinging.

  21. arXiv cs.RO62

    Semantic-Geometric Task Representation for Bimanual Manipulation from Human Demonstrations to Robot Action Planning

    The authors propose a semantic-geometric graph task representation that jointly encodes object identities, inter-object semantic relations, and each object's motion history, consisting of an MPNN encoder and a Transformer decoder; the encoder processes only the temporal scene graph and produces structured representations decoupled from action labels.

  22. arXiv cs.RO67

    DreamTacVLA: A VLA framework that uses a tactile world model to predict future contact

    DreamTacVLA proposes a framework that grounds VLA models in contact physics, learning contact dynamics by predicting future tactile signals. The model uses hierarchical perception, taking high-resolution tactile images as micro-vision inputs combined with wrist-camera local vision and third-person macro vision, and uses a Hierarchical Spatial Alignment (HSA) loss to align tactile tokens with their spatial counterparts in the wrist and third-person views.

  23. arXiv cs.RO62

    CLEAR-IR: Reconstructing Infrared Images into RGB-Style Images to Support Robot Perception in Dark Environments

    CLEAR-IR proposes a dual-stream architecture that reconstructs clean, RGB-style images from infrared images containing structured-light emitter patterns, allowing vision pipelines originally trained on RGB to keep working when no visible light is available. Inspired by DeepMAO, the architecture was evaluated on object detection, ArUco marker detection, and visual SLAM, covering scenarios ranging from well-lit to completely dark; in complete darkness, passive RGB enhancement methods fail because the sensor receives no signal. The paper has been accepted to IEEE/RSJ IROS 2026.

  24. arXiv cs.RO59

    Study: Active sampling with vision-based tactile sensors for hardness classification

    The paper investigates active sampling strategies based on vision-based tactile sensors (VBTS) for sample-efficient hardness classification, using information-theoretic measures of model uncertainty. The authors evaluated three probabilistic classifiers, two active sampling strategies, and a random sampling baseline on a hardness classification task using their own VBTS dataset collected by a robot and a previously published dataset, and also conducted a human study on hardness classification with the same objects. Results show that the active sampling methods using variance and entropy as uncertainty measures performed best, reaching 90% accuracy, compared with human accuracy of 48%.

  25. arXiv cs.RO62

    Research team builds egocentric tennis dataset and evaluates vision models' spatiotemporal prediction capabilities

    The research team collected an annotated egocentric tennis dataset and trained and evaluated image encoders, video encoders, and vision-language models on seven spatiotemporal prediction tasks. The best models achieved current-frame localization errors of 0.14 m for the egocentric player and 0.54 m for the ball, with trajectory prediction errors of 0.34 m and 0.91 m respectively within 1.1 seconds, and a mean error of 0.14 seconds for contact-time prediction. Auxiliary depth information notably improved 3D ball localization, while longer image histories did not improve ball trajectory prediction; the team also proposed a conditional diffusion model that generates full-body tennis kinematics from egocentric video at real-time rates.

  26. arXiv cs.RO60

    JAM: A Unified Action Model Framework for VLA and WAM

    The authors propose the Just Action Model (JAM) framework, which breaks down the differences between VLA and WAM into three separable choices: backbone initialization, the loss objective on future observations, and whether that loss mechanism is retained at inference. JAM unifies both into a structure of a pretrained visual backbone plus an action expert, trained with an action loss and a foresight loss.

  27. arXiv cs.RO55

    MCP-Net: A Real-Time Coronary Perception Network and Topological Analysis for Robotic Guidewire Navigation

    The authors propose a simulation-driven framework centered on a multi-task coronary perception network, MCP-Net, for robotic guidewire navigation, integrating coronary vessel segmentation, vascular topology extraction, and roadmap generation. The method generates simulated fluoroscopy images and simulation-derived ground truth from three-dimensional coronary CT angiography, including simulated guidewires, stenotic lesions of varying severity, and diverse vascular geometries. MCP-Net jointly performs vessel segmentation, guidewire-tip localization, and stenosis localization from a single fluoroscopic frame.

  28. arXiv cs.RO58

    Study Reveals Effect of Policy Standard Deviation on Loss of Plasticity in Online Reinforcement Learning

    The paper analyzes loss of plasticity (LoP) in isotropic Gaussian policies within Proximal Policy Optimization (PPO), noting that the policy standard deviation σ significantly affects the extent of LoP, with theoretical and experimental support. The experiments include LoP benchmark comparisons and real-world policy deployment on a 7-DoF robotic arm; the authors also use these findings to explain why common LoP mitigation methods are effective.

  30. arXiv cs.RO64

    ThinkWorld: A VLA Learning Framework That Reinforces Chain-of-Dynamics Reasoning with World Imagination

    ThinkWorld is a vision-language-action (VLA) learning framework that represents each action thought as a Chain-of-Dynamics (CoD), a sequence of discrete motion tokens that captures the intermediate dynamics connecting the current world to a future state and serves as a common action representation across embodiments.

  31. arXiv cs.RO75

    In-hand object pose reaching strategy and safety filter for the LEAP dexterous hand

    The research team proposed a training recipe for in-hand object pose reaching with the LEAP dexterous hand. It trains in simulation without human data and, after system identification, transfers zero-shot to the real robot, using either motion capture or a single wrist-mounted monocular depth camera. In a real-robot evaluation with 4 objects and 4,800 unseen targets over 13.3 hours, the state-based policy reached 95.2% of targets within 2 cm and 0.4 rad; adding a learned residual safety filter with adjustable conservativeness reduced drops by 75% while still reaching 91.5% of targets.

  32. arXiv cs.RO63

    EgoThumb: A Vision-Driven Supernumerary Robotic Thumb for Autonomous Grasping and Holding Assistance

    EgoThumb is a fully open-source, vision-driven supernumerary robotic thumb that uses a visual imitation learning policy to map first-person images from a palm-facing camera directly to continuous joint positions, enabling autonomous grasping and holding assistance. Its compact dual tendon-driven finger relocates the actuation unit to the forearm to reduce distal weight and support precise active joint control. The authors state that the design generalizes across different users and environments, that both hardware and software are open-sourced, and that the paper has been accepted to IROS.

  33. arXiv cs.RO71

    Research team uses differentiable simulation to learn quadrotor dynamic narrow-gap traversal from scratch

    The research proposes a differentiable moving narrow-gap modeling method that gates centering, alignment, and smooth collision terms by signed distance to a time-varying gap plane, turning sparse traversal events into dense pathwise objectives, and integrates this into a visual-feature-to-command policy trained in a single stage from random initialization, without a teacher, planner, or distillation. In simulation, path gradients clearly outperform score-function gradients, reaching a 99.3% success rate after a few minutes of training on a single GPU. The policy transfers directly to a real quadrotor using only onboard vision and inertial measurements, successfully traversing the dynamic narrow gap in 8 of 10 real-world trials.

  34. arXiv cs.RO58

    CRS: A Joint-Constraint-Referenced Evaluation Method for Articulated Object Manipulation

    The authors propose Constraint-Referenced Strategy (CRS), which projects the end-effector twist commanded by a policy onto the object's joint axis, separating progress along the joint direction from off-constraint waste. On 74 simulated articulated objects, binary success rate only shows a lower transfer failure rate without attribution, whereas the two CRS channels give each a distinct signature.

  35. arXiv cs.RO65

    ARC: Using Human Corrections to Guide Residual Correction of Robot Policies

    The paper proposes Adaptive Residual Correction (ARC), which learns residual corrections via reinforcement learning on top of a frozen imitation policy, using sparse human corrections to decide when to activate residual control and in which direction of the residual action space to explore. On three precision manipulation tasks with a real robot, ARC raises average task success from 33.3% to 75.0%; the human corrections define 2-3 dimensional task-relevant subspaces of the seven-joint residual space, retaining 96.0-97.7% of the squared correction magnitude.

  36. arXiv cs.RO80

    Rendering-Free Event Frame Simulation Accelerates Reinforcement Learning for Autonomous Robot Navigation

    The authors propose an analytic event simulator that synthesizes event streams directly from scene geometry and ego-motion without image rendering, generating 64x64 event observations in just 17 microseconds on a single GPU, over 400 times faster than state-of-the-art rendering pipelines. Reinforcement learning training for an event-camera obstacle-avoidance policy based on this simulator converges in under 30 minutes, three orders of magnitude faster than training on rendered events, and transfers zero-shot to ray-traced event simulation in Isaac Sim.

    Why it matters: This work uses analytic event simulation to bypass image rendering, cutting event-camera reinforcement learning training time to under 30 minutes, and completes real-world deployment on a 56-gram drone.

  37. arXiv cs.RO60

    StabBot: Robotic Pipe Stabbing with Learned Stereo Vision and Residual Reinforcement Learning

    The StabBot robotic stabbing system uses a pendulum Kalman filter to track pipe joints in stereo images, approaches at matched speed, and confirms the final few millimeters by touch. In 200 randomized jobs, StabBot completed every stab, versus 74% for a conventional fixed guide; median contact speed fell from 18.4 cm/s to 2.3 cm/s, and median peak head-pipe force fell from 11.3 kN to 1.4 kN.

  38. arXiv cs.RO55

    Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use

    The study measures, on a simulated bimanual tool-use task, which failure causes each sensing channel lets a diagnoser identify: a tool slipping out of its socket can be read on the tool-hand side; which hand was late requires information from both arms, while a sensor on one hand can only read that hand's own delay and part of the other hand's delay; and when the tool hand is late, passive sensing cannot determine whether the push force was also insufficient.

  39. arXiv cs.RO70

    World Dreamer v5-lite: Multi-Robot World Action Model with a Single Dreamer

    World Dreamer v5-lite is a multi-robot world action model that predicts separate action sequences for each robot through a shared world state. The model builds the world state in workspace coordinates from calibrated features of a pretrained video encoder, and its spatial size does not change as views are added; a vision-language model provides scene context and one query per robot, and the queries read the shared state and exchange pose-aware information while their action streams remain separate.

  40. arXiv cs.RO58

    Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos

    Ego-CEVR proposes a data-native hierarchical visual representation learning framework that assigns distinct learning objectives to global and interaction representations. The global module learns object relations, task state, and temporal progress through future prediction on egocentric video; the interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. Two residual paths feed both representations into the action policy simultaneously, and the global video module can be pretrained independently on large-scale human egocentric videos and videos from multiple robot embodiments, without robot action labels or paired wrist observations.