Skip to content
Only Research
Oct 26Mon
  1. arXiv cs.RO63

    Deep Learning Warm Starts Accelerate Trajectory Optimization for Astrobee on the International Space Station

    A research team completed an on-orbit demonstration on the International Space Station's Astrobee free-flying robot of machine learning-based warm starts to accelerate trajectory optimization, using a neural network to predict initial values for a sequential convex programming (SCP) solver, which then enforces safety constraints. In scenarios including rotational dynamics, the trained network reduced the number of solver iterations required for convergence by 60%, and by 50% in obstacle scenarios drawn from the training distribution. The work was published at iSpaRo 2025.

  2. arXiv cs.RO58

    FH-HBTS: A Haptic Bilateral Teleoperation System for Free-Hand Dental Procedures

    The authors propose FH-HBTS, a haptic bilateral teleoperation system for free-hand dental procedures, featuring a mechanical end-effector compatible with standard clinical tools and equipped with an endoscopic camera. The system improves the operator's sensory awareness and motor accuracy through motion and force correspondence between the operator and the robot, and limits interaction forces by scaling the motion references provided to the admittance controller based solely on measured contact forces, thereby achieving force limitation without prior knowledge of the environment. Technical evaluations and in vitro preclinical studies on dental models showed that the system improves the naturalness, safety, and precision of teleoperation.

  3. arXiv cs.RO65

    SceneBloom: An Agentic Indoor Environment Generation Framework for Embodied Learning

    SceneBloom proposes an agentic framework that combines visual layout sampling with contract-guided 3D compilation to generate realistic and diverse simulated indoor environments. Its hierarchical sampler uses diffusion priors to generate furniture layouts within functional zones, and the compiler then performs geometric fitting and local pose correction based on relations extracted from each proposal.

  4. arXiv cs.RO67

    RobustRLlib released: unifies 17 robust reinforcement learning algorithms and provides evaluation across 273 perturbation configurations

    RobustRLlib is an algorithm-centric library and benchmark for robust reinforcement learning that integrates 17 representative online, offline, and safe reinforcement learning algorithms under a unified interface. Its evaluation platform covers a variety of real-world shifts as well as humanoid robot and vision-language-action tasks, and it presents the capability distribution of current robust reinforcement learning algorithms across 273 perturbation configurations, identifying which scenarios remain effective and where substantial robustness gaps persist. The project provides open-source code, a project page, and tutorials to facilitate reproduction and extension.

  5. arXiv cs.RO60

    SMM-AC: Regularized reinforcement learning improves sim-to-real transfer

    The authors propose SMM-AC, a regularized reinforcement learning algorithm based on the soft mellowmax value function. On MuJoCo continuous control benchmarks it is comparable to baselines such as SAC, and on real robots it achieves higher returns while being more robust to perturbations in observations, the environment, and dynamics. The study compares the sim-to-real transfer performance of commonly used reinforcement learning algorithms and finds that regularization improves policy robustness to the sim-to-real gap and to real-world environment changes.

  6. arXiv cs.RO68

    Retiming Robot Demonstrations with a Learned Speed Prior

    The study proposes learning a speed prior from large-scale demonstration data: given a task instruction, a motion path, and images along the path, it predicts a distribution of execution speed along the path and uses a selected quantile as a unified clock to retime each demonstration. The method fits the speed prior on robot teleoperation data and on human handheld-gripper demonstrations, retimes robot demonstrations to the handheld pace, and then fine-tunes a vision-language-action policy; at execution, a real-time optimal-control layer tracks the predicted action chunks under robot constraints.

  7. arXiv cs.RO47

    CEMIR vs. FISER: A Comparative Study of Reasoning-Based and Enrichment-Based Approaches to Pragmatic Intent Recognition

    The study compares FISER, which is based on structured social and embodied reasoning, with CEMIR, which performs multimodal enrichment by synthesizing narratives and photorealistic images from text prompts, on the HandMeThat v2 benchmark, covering four large language models from Qwen2.5-Omni-3B to GPT-5.

  8. arXiv cs.RO58

    Study Reveals Flaws in Humanoid Robot Safety Filters; SIREN Can Automatically Generate Collision Attacks

    The paper points out that the mathematical guarantees of humanoid robot safety filters hold only when parameters are configured correctly, while the parameters depend on the specific task and environment and may fail after deployment. The authors propose goal insertion attacks and develop an automated tool, SIREN, which identified attacks in all vulnerable scenarios in the SPARK dataset, three times the number found by baseline methods, and triggered collisions on a real humanoid robot.

  9. arXiv cs.RO61

    SAGE: Work-zone path prediction under temporary traffic control combining vision-language and geometric frameworks

    The authors propose the SAGE (Semantic-to-Action Grounded Execution) framework, which uses trajectories output by a vision-language model (VLM) as preferences over complete geometric candidate paths, and combines each candidate's geometric score with its agreement with the VLM trajectory through rank fusion.

  10. arXiv cs.RO42

    StackGen3D: Generating Stable 3D Block Stack Structures from Multiview Silhouettes

    The StackGen3D framework generates stable block assembly structures from a given block inventory and three orthographic silhouette images. It first uses a physics-informed graph neural network (PI-GNN) to model support relationships between blocks and to curate a dataset of stable structures, then trains a conditional flow-matching model to learn the multimodal distribution of block configurations that match the target silhouettes. In preliminary experiments, Best-of-10 selection raised physical stability from 45.3% to 72.4%, with a silhouette IoU of 0.970.

  11. arXiv cs.RO58

    PnP*: A Certifiable Solver for Perspective-n-Point

    The authors propose PnP*, which they describe as the first practical certifiable solver that guarantees efficient computation while returning a globally optimal camera pose. The paper proves that the convex relaxation is exact under moderate noise, and uses chordal sparsity patterns and quotient ring reduction to make the relaxation tractable and scalable. Numerical validation shows that PnP* returns provably correct camera poses at over 10 Hz.

  13. arXiv cs.RO72

    DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection

    DexPort is a hand-held data collection interface for high-DoF robot hands that uses the same hand hardware for both collection and deployment, recording joint target commands and measured joint states. The operator moves the hand-held interface to specify wrist motion and coordinates finger motion by selecting or adjusting skills, without needing to control each joint individually.

  14. arXiv cs.RO72

    ViT-H: Generating humanoid motions directly from video with diffusion models and post-training via tracking feedback

    ViT-H proposes a generation-plus-tracking video-to-humanoid control framework. It uses a diffusion model to generate humanoid reference motions directly from monocular video, skipping human motion reconstruction and retargeting, and uses motion-tracking results to post-train the generator with direct preference optimization. The authors also present an autoregressive variant that enables real-time humanoid video teleoperation with sub-400 ms latency. Experiments on the Unitree G1 show that ViT-H outperforms existing reconstruction-retargeting-tracking pipelines in motion quality and executability, and that performance improves with model size and with pre-training and post-training data scale.

  15. arXiv cs.RO62

    OCT-Guided Soft Tissue Geometric Modeling: 3D Representations for Data-Driven Closed-Loop Shape Control

    The study proposes an OCT-guided data-driven framework that learns soft tissue shape control from user demonstrations and executes it with closed-loop OCT feedback. In 12 configurations and 576 phantom trials, the Explicit policy using residual-conditioned voxel point clouds achieved Shape RMSE of 0.534 mm on the depth task and 0.443 mm on the directional task; full-volume Diffusion with residual conditioning reduced directional error from 0.748 mm to 0.555 mm.

  16. arXiv cs.RO60

    Backtrack: Lightweight Failure Detection and Backtracking Recovery for Frozen VLAs

    The authors propose Backtrack, a lightweight failure detection and recovery mechanism for frozen VLAs that requires no recovery demonstrations, policy retraining, or additional inference models. It uses a failure prediction head that adds only 0.48-0.88% to the parameter count to read the VLA's internal representations; when the predicted failure probability exceeds a threshold, it pauses the policy, backtracks the robot arm's recent motion, and hands control back to the original policy for a retry.

  18. arXiv cs.RO32

    How Human Corrections Travel Across Robots: Safety Issues in Federated VLA Adaptation

    Regarding the prospects of combining federated learning with VLA policies, the research points out that sharing human-correction adaptations across tasks, environments, people, and robot embodiments carries safety risks: a shared update that raises average task success may suppress rare safety interventions or produce unsafe responses on another robot.

  19. arXiv cs.RO72

    NimCritic: An Action-Aware Quantification Model for Long-Horizon Manipulation Progress Prediction

    The study proposes an action-aware quantification method that uses a vision-language model to decompose long-horizon manipulation trajectories into object-level subtasks and assign weighted action credits, and builds a training video dataset of more than 800 hours plus the LOOM benchmark, which contains 19,618 human-calibrated evaluation frames.

  20. arXiv cs.RO60

    VERID: A semantic digital twin infrastructure for agentic self-play in automated laboratories

    The authors propose VERID, a persistent, high-fidelity semantic digital twin for automated laboratories that provides AI agents with four core services: queryable introspection, time-compressed simulation, comprehensive trajectory persistence, and a safe execution framework. Three case studies using mixtures of water-soluble dye and deionized water validated its optical and fluidic foundations, demonstrating pre-detection of critical protocol errors, in-run stateful error recovery, and post-hoc identification of subtle physical faults.

  21. arXiv cs.RO60

    AWARE: A world-model framework that predicts robot dynamics and identifies anomalies using only external cameras

    AWARE is a world-model framework that relies solely on external camera monitoring, performing anomaly detection through system dynamics estimation and state prediction without onboard sensors such as motor encoders. On 240 real-world robotic crane trajectories (including payload collisions and motor degradation), its anomaly classification improves by 26% over the strongest baseline; the inferred dynamics parameters can also be decoded to localize the source of faults, achieving 88% accuracy in localizing degrading motors.

  22. arXiv cs.RO40

    Conditional Neural Processes for Self-Supervised Goal Discovery in Action Imitation

    The study evaluates whether Conditional Neural Processes can autonomously learn action representations that are informative about underlying goals. It finds that adding an attention mechanism to the latent space enables robust goal representations to form early in the observation of an action sequence. The authors accordingly suggest the model could serve as a general self-supervised learner for robots. The work was presented as a poster at the IROS 2023 Workshop on World Models and Predictive Coding in Cognitive Robotics.

  24. arXiv cs.RO64

    Natural-Language to Nav2: An Operational Policy Interface for Costmap-Based Navigation

    The paper proposes a natural-language policy interface that connects operator instructions to ROS 2 Nav2: a local large language model converts instructions into JSON policies specifying destinations, spatial constraints, and temporal or event conditions; deterministic checks then decide whether to accept them, and three costmap adapters inject them into the existing navigation stack without changing the planner or controller.

  25. arXiv cs.RO62

    OWLS: Open-world language-conditioned speed fields for zero-shot rover navigation

    The authors propose OWLS (Open-world Language-conditioned Speed Fields), which plans directly over the dense feature maps of a vision-language model (VLM): given a description of traversable terrain, the VLM scores every pixel of a single camera image, and OWLS projects these scores onto the ground, adapts them to the platform with two morphological operations, and uses them as the speed field of a minimum-time planner, so that terrain appearance and obstacle spacing jointly shape a single trajectory.

  26. arXiv cs.RO58

    UAV-SwarmAgent: Multi-UAV reconnaissance and task allocation via code generation and vision-language reasoning

    UAV-SwarmAgent is a hierarchical multi-agent system in which an operator issues a mission in one natural-language sentence, and a ground-station agent generates and executes code that turns it into routes for each UAV. The system uses a vision-language model to locate inspection targets on an aerial map, balances them among N UAVs by route length, and then worker agents equipped with lightweight decision models verify incidents.

  27. arXiv cs.RO53

    HGWM: Zero-Shot Object Navigation Using Hierarchical Graph-Guided World Models and Graph Matching

    The authors propose HGWM, which combines an LLM-generated room-anchor prior with persistent spatial-semantic memory to select search directions in zero-shot object navigation. On HM3D v0.1, it reports 62.6% success rate and 31.5% SPL, versus 58.1% and 31.2% for WMNav; on MP3D, 46.8% and 17.9%, versus 45.4% and 17.2%.

  28. arXiv cs.RO38

    Yonghyeon Lee releases v1 of compiled lecture notes "Optimization, Optimal Control, and Reinforcement Learning for Robotics"

    The v1 lecture notes "Optimization, Optimal Control, and Reinforcement Learning for Robotics," compiled and edited by Yonghyeon Lee, have been released on arXiv, emphasizing the mathematical connections and practical applications among the three. The notes were written through human-AI collaboration and are a compilation of existing material rather than new research results; they are currently a preliminary version with no figures or graphs, and revisions and expansion are planned.

  29. arXiv cs.RO42

    Facultative Swarming: A Capability, Not a Category, for Multi-Robot Systems

    This research proposes a unified control framework that shifts multi-robot systems from "obligate swarms" constrained to pure self-organization toward "facultative swarms"—hybrid architectures that operate with centralized planning when conditions are permissive and degrade smoothly into decentralized swarming under environmental stress. The authors argue for redefining swarming behavior as a dynamic capability rather than a fixed system category, thereby exploiting centralized optimality when conditions allow and the flexibility of self-organization when they do not.

  30. arXiv cs.RO71

    From Freehand Demonstrations to Autonomous Robotic Ultrasound: An Anatomy-Anchored Imitation Learning Framework

    The study proposes an anatomy-anchored imitation learning framework that combines anatomy-guided probe placement with learned ultrasound navigation. Across 16 unseen participants, task success rates for standard-view acquisition of the kidney, gallbladder, and liver and for radial artery scanning were 96%, 94%, 88%, and 94%, respectively.

  32. arXiv cs.RO60

    R²E: A multi-role LLM evolutionary framework for automated reward function design

    The authors propose R²E, a framework for automated reward function design that uses explicit role specialization to mitigate insufficient exploration and optimization instability in LLM-based reward design. The framework includes a novelty explorer responsible for expanding the search space, a guardian responsible for conservative refinements to improve training stability, and an evolver that recombines high-performing rewards through crossover and mutation, with a dynamic scheduling strategy that gradually shifts the search from broad exploration to focused exploitation. R²E also introduces a global elite pool that retains the best-performing rewards to guide subsequent iterations. Experiments on multiple robotic tasks show the effectiveness of the method.

  33. arXiv cs.RO58

    GTB: A Trajectory Bundle Graph Motion Planning Method for Nonlinear Dynamics

    The authors propose the Graphs of Trajectory Bundles (GTB) method, which combines sampling-based motion planning, trajectory optimization, and graph search for motion planning of systems with nonlinear and even black-box dynamics. Based on a graph-of-convex-sets decomposition of the workspace, the method iteratively constructs a graph of trajectory bundles that captures the local dynamics at each vertex, then solves a shortest-path problem on the graph to obtain trajectories that satisfy the system dynamics and constraints such as collision avoidance and safety in the limit. The authors claim it can handle highly nonlinear, unstable, and even black-box dynamics with complex constraints.

  34. arXiv cs.RO62

    LogLook: Execution-Log Memory for Frontier-Model Robot Planners

    LogLook proposes an execution-log-style memory that lets frontier vision-language model planners record executed actions and earlier notes, and selectively review image frames based on expected changes. With the benchmark's built-in motion planner, it matches the success rate of note-only memory while using 43% fewer tokens and less than half the images, and it causes notably fewer failures due to memory or observation errors; under direct control, it solves 93.5% of the RoboMME test set.

  35. arXiv cs.RO62

    DynaSlots: A Slot-Level 3D Dynamics Pretraining Framework for Robot Manipulation

    DynaSlots is an action-label-free 3D visual pretraining framework that decomposes each point cloud frame into a small set of temporally consistent slots and models dynamics at the slot level rather than the scene level. A temporal slot propagation module keeps slot assignments consistent across frames; an inverse dynamics model extracts a latent action for each slot from feature differences; and a diffusion-based forward dynamics model, paired with a self-supervised interaction head, predicts each slot's future state and learns pairwise proximity and motion relationships.

  36. arXiv cs.RO62

    MAST-VLA: Multi-Skill Robot Manipulation with Skill-Conditioned Experts and Trajectory-Affordance Priors

    MAST-VLA proposes a dual-conditioning framework that introduces skill-conditioned LoRA Mixture-of-Experts adapters into the LLM backbone and converts predicted affordance heatmaps into a patch-level trajectory-affordance spatial prior for the action head. In the CALVIN ABC→D long-horizon multi-skill cross-scene setting, it achieves an average completion length of 4.65 over five-subtask sequences, the best performance in that setting. Real-robot multi-skill sequence experiments further validate its deployability under real perception and control noise. The paper has been accepted to CoRL 2026.

  38. arXiv cs.RO60

    From Tabletop to Walking Robot: Deploying a VLA Model on a Legged Manipulation Robot

    The research team deployed a VLA model on a Unitree Go2 quadruped equipped with an AgileX Piper robotic arm, achieving closed-loop mobile manipulation, and reported choices such as model variant, action horizon, and demonstration conventions that made real-robot deployment feasible. For the pick-and-place task, expanding demonstration coverage reduced held-out end-effector error by 30%, and adding secondary skills such as pushing introduced new behavior modes without degrading baseline performance. Closed-loop real-robot experiments showed that dataset composition affects task success.

  39. arXiv cs.RO57

    RoboCode: Learning Reusable Action Primitives for Robot Manipulation

    RoboCode learns short-horizon motion patterns that can be reused across tasks through a shared soft-gated primitive action interface, representing action chunks as soft compositions of multiple learned primitives rather than a single discrete code. The interface is learned on action chunks pooled across multiple tasks and then frozen, after which task-specific state encoders are trained to map into the shared latent space. On five ManiSkill tasks, it achieves an average closed-loop success rate of 75.6%, above ACT's 68.8% and Diffusion Policy's 41.6%, with an average action chunk reconstruction MSE of 0.0187.

  40. arXiv cs.RO58

    Do VLAs Know They Will Fail? Interpretable Success and Failure Patterns in Physical AI Systems

    Research shows that sparse autoencoder (SAE) representations of OpenVLA activations contain outcome-predictive information and can predict task failure before the robot reaches an unrecoverable state. The authors also identify individual SAE features associated with specific stages of success or failure, and uncover recurring behavioral patterns by localizing temporal intervals where these features show significant activation changes. The team further builds a VLM-based interpretation pipeline to generate behavioral descriptions of these patterns and validates their correspondence to feature activations on held-out intervals.