Skip to content
#Robot models and learning
Oct 27Tue
  1. arXiv cs.RO71

    RoboGene: A Multi-Arm Morphology-Diverse Task Generation Framework for VLA Pre-Training

    RoboGene is an agentic task generation framework for single-arm, dual-arm, and mobile robots that automatically produces diverse manipulation tasks before VLA pre-training. It combines diversity-driven sampling, agentic self-reflection to assess task quality, and human-verified feedback memory for execution-grounded validation and continual improvement. It collected 18,000 trajectories and introduces task feasibility and diversity metrics.

  2. arXiv cs.RO70

    FlowHOI: Flow-Matching-Based Semantics-Grounded Generation of Hand-Object Interaction Sequences for Dexterous Manipulation

    FlowHOI is a two-stage flow-matching framework that generates semantically grounded hand-object interaction sequences, including hand poses, object poses, and contact states, conditioned on first-person observations, language instructions, and 3D Gaussian splatting scene reconstructions.

  3. arXiv cs.RO71

    UniManip: An Open-World Robotic Manipulation Framework Based on Agentic Operational Graphs

    UniManip proposes a unified framework centered on the Agentic Operational Graph (AOG) for task orchestration, state tracking, execution verification, and closed-loop failure recovery in open-world robotic manipulation. The method grounds high-level semantic intent into executable primitives through a physically grounded scene graph, and combines a safety-aware local planner to generate collision-free trajectories. It achieves state-of-the-art performance on LIBERO-Pro, and in real-world experiments its task success rate is 22.5% and 25.0% higher than leading VLA models and hierarchical baselines, respectively.

  4. arXiv cs.RO58

    CRAFT-E: An Interpretable and Editable Neuro-Symbolic Framework for Affordance-Grounded Object Retrieval

    CRAFT-E is a modular neuro-symbolic framework that combines an LLM-derived verb-property-object knowledge graph with CLIP-based visual alignment and energy-based grasp reasoning for affordance-grounded object retrieval. The system provides interpretable grounding paths, incorporates grasp feasibility into affordance inference, and supports graph edits without retraining.

  5. arXiv cs.RO71

    Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models

    The study proposes an automatic framework for generating robotic planning and execution failure data, builds the GuardianFail-36k failure reasoning dataset based on RLBench and BridgeDataV2, and trains Guardian, a multi-view reasoning vision-language model. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and the newly introduced UR5-Fail. When combined with LLM-based manipulation policies, it improves task success rates in both simulation and real-world deployment. Code, data, and models are publicly available.

  6. arXiv cs.RO62

    A Planning-Level Intention Prediction Framework for Mobile Robot Shared Control

    The paper proposes a shared control framework for mobile robots with planning-level intention prediction, using an intention domain to constrain path replanning and jointly modeling intention-domain prediction and replanning as a Markov decision process solved via deep reinforcement learning. The authors also design a Voronoi-based human trajectory generation algorithm that allows the model to be trained entirely in simulation without human participation or demonstration data; simulations and real-world user studies show the method reduces operator workload and improves safety without degrading task efficiency.

  7. arXiv cs.RO38

    Toward Physical Recursive Self-Improvement: A Call for Body-Control Co-Design for Humanoid Robots

    A NeurIPS 2026 Position Paper proposes body-control co-design to achieve physical Recursive Self-Improvement (RSI) in humanoid robots: using experience from the current body and policy to recursively improve both morphology and behavior at the same time. The paper discusses co-design methods grounded in strategic exploration, Sim2Real transfer, and meta-policy learning, argues for their necessity from methodological, application, and community perspectives, and lists open research questions ranging from short-term innovations to long-term goals.

  8. arXiv cs.RO34

    The Special Galilean Group SGal(3) and Its Applications to Several Robotics Problems

    The special Galilean group SGal(3) is a 10-dimensional Lie group whose important subgroups include the special orthogonal group, the special Euclidean group, and the group of extended poses. This work briefly describes SGal(3) and its Lie algebra, explains how the group structure supports a unified representation of uncertainty in space and time, and points out the group's potential uses for several robotics problems.

  9. arXiv cs.RO66

    OSWAM: A World Action Model with One-Step Action Generation

    OSWAM proposes a world action model that generates actions with just one diffusion step, for autonomous driving and embodied control. The authors initialize the diffusion process with imagination tokens to provide physical priors and action-conditioned representations, and introduce privileged information distillation with future conditioning, making action quality comparable to or better than direct-policy WAM and Joint-WAM, which require tens of steps. It is about 3 times faster than the direct-policy WAM baseline, with compiled inference latency as low as 40ms on an NVIDIA RTX 5090 GPU.

  10. arXiv cs.RO58

    Multi-Objective Reinforcement Learning with Routed Ensembles and Trajectory Attribution

    This paper studies policy learning in multi-objective reinforcement learning, proposing objective-conflict coefficients and using them to derive near-optimal regret bounds for online settings, quantifying the impact of objective misalignment on learning efficiency. The authors design a routed ensemble policy learning algorithm that maintains a small set of basis policies and dynamically routes trajectories to the policy with the highest potential reward; they also propose a kernel surrogate trajectory attribution algorithm that estimates how trajectory feedback signals affect the policy learning of other trajectories.

  11. arXiv cs.RO62

    OpTTA: Test-time adaptation of world models for planning, not just prediction

    The paper proposes OpTTA, which adds the planner's goal-reaching cost term to the test-time adaptation objective for world models, keeping the model optimistic about reaching the goal. Under visual and environmental shifts in manipulation, reaching, and navigation tasks, when only the predictor is adapted, OpTTA raises the final planning success rate by an average of 13.5 percentage points, compared with 6.9 percentage points for AdaJEPA, which uses only the prediction objective; when the encoder is also adapted, the two achieve 13.5 and 7.5 percentage points, respectively.

  12. arXiv cs.RO62

    High-resolution tactile observation makes hidden racket-control mechanisms learnable

    The research team used an 864-node tactile-inertial glove to synchronously collect more than 14,000 strokes at 100 Hz from over 70 participants across five skill levels, and proposed a mechanics-based two-layer framework that infers the control driving the racket and stabilizing the grip from tactile history and reconstructs its spatiotemporal force realization.

  13. arXiv cs.RO72

    Video World Models Need Predictable Latents

    This paper proposes making the latent space of video world models balance predictability with pixel reconstruction accuracy, achieved through temporal objectives for representation learning paired with a past-conditioned decoder architecture. Under the same training budget, inference cost is reduced by up to 5x, training speed increases by more than 7x, and pixel and perceptual errors in out-of-distribution and long-horizon prediction drop by 15-24%. The authors argue that temporal predictability should be an important design principle for video world model latent spaces and provide a concrete method for incorporating it into autoencoder training.

  14. arXiv cs.RO49

    Whole-brain recordings in fruit flies and a neuromechanical digital twin: how intrinsic goals in hierarchical exploration capture future volitional action

    The study combines whole-brain recordings in fruit flies with a neuromechanical digital twin to test whether agents trained with intrinsic motivation algorithms can reproduce the fly's spontaneous exploratory behavior in a virtual tethered-ball experiment. Agents select actions from a discrete library of low-level motor primitives and are trained without external reward using 20 intrinsic motivation algorithms; among them, DIAYN came closest to the fly in behavior and neural prospective encoding, and reproduced the additive and subtractive rules of interhemispheric kinematic encoding.

  15. arXiv cs.RO72

    FLORE: Learning Force-Reactive Humanoid Loco-Manipulation

    The research team proposes the FLORE framework, which uses two force-sensing UMI-style handheld grippers and wearable trackers to collect robot-free demonstrations, and trains a force-reactive diffusion policy that takes vision, proprioception, and measured force wrenches as input and outputs actions and per-hand force commands. A companion hybrid compliant whole-body controller uses integral force error from real force/torque sensors to achieve single-hand force tracking while maintaining balance. In real-robot tests on three contact-rich tasks, the force-reactive policy achieved an 80% success rate, compared with 26.7% for a position-only policy under the same controller.

  16. arXiv cs.RO71

    Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization

    This paper proposes a smooth analytic gradient for frictional contact that replaces the exact contact sensitivity only in the backward pass, leaving the forward hard-contact solve and the hardware-feasible trajectory unchanged. A policy trained on 100,000 samples in under 2 minutes on a desktop PC transfers to hardware without fine-tuning; under a trotting policy the robot withstands pushes and traverses gravel and slippery ground. In trajectory optimization, the gradient finds contact-rich solutions that the exact gradient and the Signorini relaxation gradient miss.

  19. arXiv cs.RO60

    TrySee: Modular Perception-Enhanced VLM Closed-Loop Robot Task Planning for Specialized Domains

    TrySee proposes a modular closed-loop robot task planning pipeline that combines general-purpose VLM reasoning with domain-specific perception and existing robotics components, without requiring VLM fine-tuning or end-to-end demonstrations. The system uses reactive cyclic planning and adds domain-specific modular vision preprocessing to provide online grounding, while a dynamic preprocessing selection mechanism improves scalability.

  20. arXiv cs.RO53

    FISRF: An Adaptive Reflection-Guided Framework for LLM-Based Robot Task Planning

    FISRF proposes an agent-and-mentor architecture that estimates a robot's Zone of Proximal Development (ZPD) in task planning and adaptively selects reflection strategies and mentor guidance, mitigating the accumulation of erroneous experience during early interactions. Experiments in ALFWorld and on real-world robotic platforms show that the method reduces average error-correction rounds by 37% and redundant reasoning by 25.1%, while improving overall task success rate by 8%.

  21. arXiv cs.RO58

    FERRET: Using Environment Edits as a Corrective Signal for Reward Learning

    The authors propose environment edits as a complementary form of feedback for robot corrections: with the robot's motion held constant, a human edits the scene so that the only explanation for the edit is the moved object. They introduce FERRET (Feature Learning from Environmental and Robot-Relational Edit Traces), which learns preferences from both robot corrections and environment corrections, and they show that environment edits can disambiguate features and learn preferences that robot corrections alone cannot.

  22. arXiv cs.RO52

    From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Intent

    The study proposes a multimodal framework that uses large language models and vision-language models to translate human purposes expressed in natural language into robot missions and drives, aligning the behavior of open-ended learning autonomous robots with human intent. The system has three layers—semantic alignment, mission generation, and drive modeling—forming a bidirectional interaction loop in which human input defines high-level purposes, the robot expands them into executable missions, and feedback maintains alignment. Simulation and physical robot experiments show that the pipeline can generate effective missions and behaviors.

  23. arXiv cs.RO67

    3D Gaussian Splatting-Based Precise Reaching for Wrist-Camera VLA

    This work proposes a 3D Gaussian Splatting (3DGS)-based data generation pipeline for adapting wrist-camera VLA policies to fine reaching tasks. The method reconstructs the real workspace and renders wrist-camera observations from robot-aligned poses, pairing them with smooth, structured reaching trajectories for VLA fine-tuning, and adds an instruction-conditioned reach verifier that judges termination before physical contact.

  24. arXiv cs.RO60

    Context-Aware Generation and Modulation of Expressive Motion Behavior Using Multimodal Foundation Models

    This research proposes a method to automatically generate and modulate complex full-body expressive gesture sequences from multimodal context descriptions spanning text, audio, and video. It implements the principles of animation in a unified way based on Dynamic Movement Primitives (DMP) and uses multimodal foundation models to generate highly context-aware parametric motion variants.

  25. arXiv cs.RO52

    RoboDoctor: A Training-Free Recursive Self-Improvement Framework for Robot Policies

    RoboDoctor is a training-free framework that, under fixed model weights, lets generalist robot policies and language-model agents adapt during execution through three revision targets: diagnosis, action execution, and persistent experience records. On 54 RoboDojo-Sim task entries, the framework achieved a five-dimension mean score of 57.10 and a native success rate of 49.44%; recorded interactions show policy switching, selective correction, and grasp revision. The authors note that these results characterize execution-time adaptation in the evaluated system and that controlled experiments are still needed to isolate the contribution of cross-episode experience.

  26. arXiv cs.RO58

    PIFR: A Benchmark for Preemptive Instruction Following and Task Resumption in Long-Horizon Embodied Navigation

    The authors propose the PIFR benchmark to evaluate preemption and resumption capabilities in long-horizon embodied navigation when a higher-priority instruction interrupts the current task sequence. PIFR includes single, repeated, and nested interruption settings, with stage-specific metrics for inserted-task completion, resumption of the original task, and completion of the full sequence. The authors also propose a lightweight candidate-ranking baseline to improve target re-grounding after interruption.

  28. arXiv cs.RO55

    RoKON: Efficient and Adaptive Data-Enabled Predictive Control for Robotic Manipulation

    The paper proposes RoKON, a data-driven predictive control method for robotic manipulation that does not require learning an explicit dynamics model. The method uses a Gaussian-kernel nonlinear predictor to represent the input-output data relationship, proposes a damped Gauss-Newton method that exploits the least-squares structure of the control problem to achieve real-time control, and introduces a hybrid buffer management strategy that incorporates informative online data while retaining relevant offline data, allowing adaptation to changes in dynamics.

  29. arXiv cs.RO63

    EmbodiedLab releases embodied agent benchmark: letting robots "act to know" in physics experiments

    EmbodiedLab is a benchmark that requires robots to "act to know": the agent receives an objective and experimental apparatus, chooses its own experimental procedure, reads instruments via camera in a physics simulation laboratory, and is evaluated on the knowledge it ultimately gains as well as the physical state required by the task.

  30. arXiv cs.RO60

    Labference: A Benchmark for Evaluating the Reasoning-Action Connection in Robotic Laboratories

    The authors propose Labference, a benchmark for evaluating how robots adjust subsequent operations based on actual physical execution outcomes, separately measuring operation selection, physical execution, and task completion. Preliminary evaluation shows that the tested VLA policies still made target-selection errors even when object transfer succeeded, while GPT-driven control, though able to select correctly, encountered execution failures and higher computational cost.

  31. arXiv cs.RO58

    RetroNav: A Self-Evolving Lifelong Navigation Framework Combining Scene Retrospection and Retry

    RetroNav is a training-free lifelong navigation framework that integrates navigation, memory accumulation, and failure reflection, achieving state-of-the-art performance on five navigation benchmarks. It uses a unified memory system to link geometric and semantic scene knowledge with historical navigation experience, distinguishing objective observations from task-dependent interpretations. After each navigation attempt, an independent evaluator determines whether a retry is needed; when it is, a retrospection agent uses scene evidence to generate a retry brief. The authors say the method will be open-sourced.

  32. arXiv cs.RO47

    The Language of Deformation: Semantic Failure Analysis as a Prerequisite for Dexterous Manipulation of Deformable Objects

    The paper proposes that semantic failure analysis is a key step toward mature dexterous manipulation of deformable objects and supports this claim with proof-of-concept experiments. Unlike rigid-object manipulation, cables, plastic parts, cloth, and similar objects exhibit diverse and hard-to-detect failure modes such as buckling, plastic deformation, pressure changes, and tearing due to differences in materials, geometry, and behavior, requiring different corrective strategies. The authors define the components and challenges of semantic failure analysis and present experimental results indicating its potential as a research direction.

  33. arXiv cs.RO72

    Zero-HLM: A humanoid loco-manipulation framework without real robot data

    Zero-HLM proposes a real-to-sim-to-real framework that turns smartphone captures and simulated source demonstrations into scalable training data for humanoid loco-manipulation. The study generated 4,800 physically validated demonstrations and achieved zero-shot transfer on three Unitree G1 tasks; GR00T N1.7 trained in simulation reached a mean full-task success rate of 66.7%, versus 46.7% with real-data training.

  34. arXiv cs.RO62

    PushAround: Physics-Informed Hybrid Search for Multi-Robot Collaborative Path Clearing

    PushAround proposes a physics-informed hybrid search method that combines geometric connectivity reasoning with short-horizon simulation to generate executable clearing plans for mobile robot teams. The method uses repeated W-clearance connectivity graph (WCCG) queries to identify frontier gaps that need widening and ranks them by estimated downstream cost, then generates direct or recursive pushing candidates, selecting obstacles, directions, contacts, and forces.

  35. arXiv cs.RO55

    Factor-Graph-Based 3D Force Estimation for Dual-Arm Manipulation of Flexible Cables

    This study proposes using factor graphs to probabilistically model deformable linear objects (DLOs), estimating the magnitude, direction, and location of 3D forces acting on them while simultaneously reconstructing the cable's shape. Because cables are thin and unactuated, strain gauges and actuation readings are generally impractical; the method uses only one type of sensing information and performs force estimation in simulation tests that add measurement noise and have the cable move along a prescribed path. The authors state that this general method can apply to a broader range of tasks.

  36. arXiv cs.RO60

    VISTA: An Active Exploration Framework for Self-Improvement in Embodied Agents

    VISTA is a meta-agent framework that replaces post-hoc replay reflection with active self-exploration to improve the harness of embodied agents. It maintains a harness graph that records the lineage of harness versions, enabling the meta-agent to retrieve past hypotheses, reuse effective mechanisms, and avoid repeating failed directions. Each round starts from a retained version and its execution evidence, autonomously generates new rollouts, probes failure and recovery behavior, and on that basis iteratively revises the harness and re-evaluates it.

  38. arXiv cs.RO62

    LECO: Learning Direct Control from Optimization for Dynamic Target Tracking

    LECO converts online trajectory optimization for dynamic target tracking into offline learning of a direct-control policy, using a differentiable kinematic model to let trajectory-level objectives directly guide control learning. Across three simulated obstacle densities, its root-mean-square linear jerk is 14.1%–31.2% lower than the best comparison method on this metric, it achieves the highest tracking success rate, and its per-cycle computation time is 4.25–5.63 ms.

  39. arXiv cs.RO58

    MMGNet: A Multimodal 6-DoF Grasp Synthesis Network for Cluttered Scenes

    MMGNet proposes a multimodal grasp synthesis network that uses hierarchical image-to-3D feature injection and rotational graspness modeling to integrate multi-scale image features into 3D representations and jointly predict grasp approach direction and in-plane rotation. On GraspNet-1Billion with RealSense data, it achieves a mean AP of 60.34, surpassing all compared methods; in real-world experiments with adversarial grasp objects, its grasp success rate is 8.46 percentage points higher than AnyGrasp. Ablation studies show that the two modules provide complementary gains. Code, models, and data will be released after the paper is published.

  40. arXiv cs.RO60

    Disturbance-separated model predictive control for airborne electro-optical gimbals

    The paper proposes a disturbance-separated model predictive control (MPC) scheme for constrained pointing control of airborne electro-optical gimbals under carrier vibration, input delay, and actuator constraints. The scheme uses an adaptive periodic observer combined with linear-regression frequency estimation to predict multi-harmonic vibrations, and a linear observer plus a bounded LSTM network to estimate aperiodic disturbances and their changes over the input delay; periodic predictions are embedded in the MPC model, while aperiodic compensation is applied outside the optimizer.