Skip to content
#Research
Oct 21Wed
  1. arXiv cs.RO60

    FastVLA: Task-conditioned and temporally adaptive visual token compression for vision-language-action models

    FastVLA proposes a training-free, task-aware, and temporally adaptive visual token compression framework to improve the closed-loop inference efficiency of vision-language-action (VLA) models. It scores tokens via task-to-vision attention and adaptively sets the compression rate based on the uniformity of the importance distribution, calibrated using an inverse Simpson index and an online sliding window.

  2. arXiv cs.RO61

    ForeTac: A Predictive Contact Guidance Framework for Generative Robot Policies

    ForeTac is a predictive contact guidance framework for contact-rich manipulation that uses a represent-predict-evaluate-guide loop: it encodes the current tactile state, predicts future tactile latent trajectories conditioned on candidate action sequences, and evaluates each trajectory with a contact-quality objective learned from successful and failed physical interactions. At inference, gradients of this objective are backpropagated through the tactile predictor to guide action generation before execution, while all model parameters remain frozen.

  4. arXiv cs.RO67

    Learning Bimanual Laparoscopic Appendectomy Without Robot-Collected Demonstrations, Validated on Live Animals

    The paper proposes an end-to-end pipeline that uses an instrument-state logger mounted on the shaft of a standard laparoscopic instrument (combining inertial, time-of-flight, and Hall sensors, with no external camera or tracker) to capture the instrument pose and jaw state during a surgeon's operation, which is then used to train a surgical robot policy and validated on live animals.

  5. arXiv cs.RO56

    arXiv paper explores using nonlinear body oscillations to generate natural quadruped gaits

    The paper proposes a proof-of-concept framework that incorporates a robot's nonlinear dynamics into the design process, enabling multi-gait locomotion to emerge from nonlinear resonances shaped by gravity, inertia, and elasticity. The authors introduce the highly compliant quadruped robot eBert, use new theoretical tools to identify six nonlinear normal modes (NNMs), and validate their existence in simulation and hardware. Using black-box optimization to determine step length, simulations show that each NNM naturally develops into a distinct gait with different speeds, and most of these largely transfer to the physical hardware.

  6. arXiv cs.RO50

    Readiness Barrier Functions: Forward-Invariant Control Authority for Overactuated Multirotor Allocation

    This paper proposes enforcing a control barrier function at the torque level via a quadratic program in the allocation null space, keeping the control authority of an overactuated multirotor forward-invariant. In simulations of a hexarotor and a fully actuated octorotor, across 300 randomized tasks, the greedy maximization scheme violated the certified lower bound in up to 51% of authority-scarce tasks, whereas the proposed filter never left the certified set, and in the hardest torque-reversal scenario the torque error was 79 times smaller; the robust variant maintained the lower bound on all 150 perturbed objects.

  7. arXiv cs.RO58

    WorldScape Policy 2.0: Steerable World Action Modeling via Reasoning-Augmented Memory and In-Context Learning

    The paper proposes WorldScape Policy 2.0, a controllable World Action Model (WAM) with reasoning-augmented long short-term memory. Its causal short-term visual memory supplies recent observations as DiT prefill to preserve local interaction dynamics, while its long-term event memory organizes historical VLM outputs into global-history, local-active, and event-boundary representations for progress-aware retrieval and to generate implicit subgoal conditions.

  8. arXiv cs.RO62

    Autonomous Data Collection and Offline Reinforcement Learning for Local Contact-Rich Manipulation Tasks

    The paper proposes a modular method that autonomously collects data in local segments of contact-rich tasks and uses offline reinforcement learning to train local policies, which are then integrated with traditional motion planning to complete long-horizon assembly tasks. Across four real-world tasks, using only 2-2.5 hours of autonomous data collection, the method achieved an average success rate of 96% on long-horizon tasks, compared with 55% for a planning-only baseline and 37% for the strongest end-to-end baseline; the local policies also reduced applied force by 49%. Performance remained at 90% in scene-level out-of-distribution scenarios, with more than 1,500 real-world evaluation rollouts completed in total.

  9. arXiv cs.RO55

    PAKE: Whole-body loco-manipulation via partial motion generation and low-level imitation control

    PAKE proposes a whole-body loco-manipulation control framework that decomposes the problem into partial reference motion generation and low-level imitation control. It introduces a Kinematic Normalizing Flow (KNF) model trained on a large-scale motion generation dataset to generate diverse partial motion references; a high-level controller navigates the KNF latent space to exploit redundant solutions, while a low-level controller learns dynamically consistent motion execution.

  11. arXiv cs.RO60

    ZeroDex: Zero-shot long-horizon dexterous manipulation via multi-view 3D-grounded VLM reasoning

    ZeroDex is a zero-shot long-horizon dexterous manipulation framework that grounds language instructions into executable 3D task plans from calibrated multi-view RGB images. It leverages the 2D grounding ability VLMs already possess and lifts it to the metric 3D quantities manipulation requires (contact points, placement targets, tool-motion anchors) through multi-view fusion; execution is driven by these 3D groundings rather than a learned policy, requiring no demonstrations or post-training for a specific task or embodiment.

  12. arXiv cs.RO57

    Flow Control: Steering VLA Actions in Real Time with Generic Inputs Such as a Keyboard

    The paper proposes a flow control method for VLA models that steers robot actions in real time through generic inputs such as a keyboard, without retraining or fine-tuning the VLA. The method writes user input into the initial condition of the VLA's flow matching action expert; because the flow is deterministic, the generated actions both reflect user intent and stay close to the VLA's own unsteered actions.

  14. arXiv cs.RO78

    Activation injection study dissects vision-language-action models

    Researchers propose an interpretability pipeline to analyze what a fine-tuned vision-language-action (VLA) policy is computing internally. On pi0.5, injecting task A's activations into the action expert while the policy views task B's scene caused 99.6% of episodes (n=1968) to execute task A's motor trajectory, indicating the injected program is bound to absolute workspace coordinates and ignores the visible scene.

    Why it matters: The study uses activation injection to break down the internal computation of VLA policies, offering a mechanistic explanation for understanding their perturbation brittleness.

  16. arXiv cs.RO45

    Learning Action and Style Priors for Joint-Torque Control in Legged Robots

    A study on deep reinforcement learning for legged robots proposes using latent actions learned from a small set of expert demonstrations as an inductive bias to help joint-level torque control policies explore more efficiently. This prior allows the agent to not be limited by the demonstration reward level and significantly improves performance on transfer tasks; when combined with style rewards for imitation, it can more closely match expert behavior. Related videos have been made public.

  17. arXiv cs.RO55

    Joints, Camera, Action! A Co-Generation Model for 3D Motion and Video for Arbitrary Skeletons

    The paper proposes a model that can simultaneously co-generate video and 3D motion for any given skeleton, along with a camera that registers that motion to the scene. The model keeps 3D motion in a separate module to support arbitrary skeletons, connects it to a pretrained video model through a small number of cross-attention layers, and uses a generative video editor to curate nearly 200K training pairs. Evaluations show that its text-driven animation of diverse skeletons is more faithful than video-first pipelines, that the motion responds to the surrounding scene, and that it also supports video-to-skeleton motion, skeleton motion-to-video, and keyframe-based 3D motion control.

  18. arXiv cs.RO46

    Planning Under Noise (PUN): A Neural Operator Planning Method Under Noise

    PUN (Planning Under Noise) is a neural operator that learns noise-aware value fields to enable motion planning with control execution noise in unseen geometric environments. It changes obstacle boundary conditions to reflecting Neumann conditions and sets a Dirichlet condition at the goal, and extends the Monte Carlo solver Walk on Stars (WoSt) to higher dimensions, learning the mapping from geometry to value field from noisy, partially converged labels.

  19. arXiv cs.RO45

    CoGaussian: A Gaussian-based open-vocabulary collaborative 3D scene understanding framework

    CoGaussian proposes an open-vocabulary collaborative semantic occupancy prediction framework that uses sparse 3D Gaussians aligned with foundation models to represent scenes as a compact representation for transmitting and fusing geometric and semantic information among multiple agents, no longer relying on dense 3D voxel annotations and no longer limited to predefined semantic categories. On the Semantic-OPV2V benchmark, the method outperforms existing open-vocabulary single-agent approaches.

  21. arXiv cs.RO38

    Distributed Platoon Control for Safe and Smooth Crossing at an Unsignalized Intersection (Part I): Stability Analysis

    A study proposes a distributed platoon controller that analyzes the asymptotic behavior of multi-vehicle traffic at an unsignalized intersection through closed-loop path modeling. It shows that under mild conditions on the controller gains, safe crossing is guaranteed during the transient phase, and in steady state all vehicles cross smoothly at a common constant speed while avoiding collisions. Unlike traditional bi-level decoupled designs on open paths, the method couples intersection control with platoon convergence analysis. Its effectiveness was validated in a high-fidelity CARLA simulator using multiple commercial vehicle models.

  22. arXiv cs.RO47

    Interruption-aware recurrent reinforcement learning improves UAS navigation safety for temporary overrides

    To address the increased collision risk when unmanned aircraft systems (UAS) resume autonomous navigation after temporary takeover by teleoperation or safety mechanisms, the study proposes three interruption-aware recurrent reinforcement learning architectures that incorporate interruption context into recurrent processing through independent feature streams, dedicated interruption memory, or gated environmental features.

  23. arXiv cs.RO56

    RoboJudge-Bench evaluates multimodal large models as judges of embodied video generation

    The paper introduces RoboJudge-Bench to assess the reliability of multimodal large language models (MLLMs) as judges of embodied video generation, covering two dimensions, physical consistency and instruction alignment, with 800 human-annotated videos from 8 source corpora and 10 video generators, spanning grippers, human hands, and dexterous hands.

  24. arXiv cs.RO45

    Giving Models a Finger: Developmentally Aligned Evaluation of Counting in Vision-Language Models

    A study of 9 vision-language models, 682 images, and 7 evaluation conditions found that models' exact-counting ranges differ by nearly two orders of magnitude (R80 from 3 to 150 dots). Using a training-free agent loop that lets models mark images to aid counting, the study found this method mostly failed as an intervention but produced auditable serial reasoning traces that help reveal how models arrive at answers. Experiments also showed that models often change answers when traversal order is reversed, and that a child-like advantage from regular arrangements appeared only under specific conditions.

  25. arXiv cs.RO47

    When Does Experience Memory Actually Help Closed-Loop LLM Driving Agents? A Controlled Evaluation

    A controlled evaluation on the HighwayEnv testbed (1Hz, with gpt-4o-mini used for the key comparisons) shows that the published gains from experience memory in closed-loop LLM driving agents mainly come from sampling luck in the warmup pool: retrieval-based memory beat No-Memory in only one of ten pools (sign test p=0.011).

  26. arXiv cs.RO55

    PanoGraphGS unifies panoptic segmentation and hierarchical 3D scene graph construction, releases NaviGS dataset

    PanoGraphGS integrates panoramic 3D Gaussian representation learning and hierarchical 3D scene graph construction into a single framework, learning semantic and instance features on shared Gaussian geometry to achieve cross-view-consistent panoptic prediction and novel-view panoptic segmentation.

  27. arXiv cs.RO55

    SynCalib: Breaking Dataset Shortcuts in LiDAR-Camera Calibration via Extrinsic-Controllable Synthesis

    The paper proposes SynCalib, a data generation framework that addresses overfitting and poor generalization caused by insufficient extrinsic diversity in training data for deep learning-based online LiDAR-camera calibration. Using omnidirectional image datasets annotated with depth ground truth, it synthesizes multimodal calibration data under strict extrinsic constraints. Experiments show that this pipeline significantly improves the generalization and robustness of learning-based calibration networks, and real-world evaluations across various camera-LiDAR configurations validate the zero-shot transfer capability of models trained solely on synthetic data; the data generation code, pretrained models, and online benchmark dataset will be made public.

  28. arXiv cs.RO44

    OmniVLoc: A Topo-metric Visual Localization System for Omnidirectional Cameras

    OmniVLoc proposes a surround-view visual localization system for omnidirectional cameras based on a topo-metric framework. It fuses global place recognition with monocular depth estimation to achieve multi-constrained localization, and solves a scale-aware pose graph and spherical bundle adjustment to recover metric-level poses. The system includes a hierarchical verification mechanism that filters erroneous outputs at each stage to improve robustness. Experiments on public datasets and a custom dataset validate its accuracy and reliability, and the source code and dataset will be released after the paper is published.

  29. arXiv cs.RO58

    arXiv releases 12-chapter, 388-page monograph 'Physical AI: Concepts, Techniques, and Challenges'

    arXiv has released a monograph titled 'Physical AI: Concepts, Techniques, and Challenges,' comprising 12 chapters, 388 pages, and 64 figures. It defines physical AI as artificial systems that perceive, model, and act upon the physical world through continuous feedback under real-world constraints, and distinguishes it from digital AI, embodied AI, cyber-physical systems, and digital twins.

  30. arXiv cs.RO55

    Latent Projection for Latent Planning: Improving World-Model Planning Costs with Latent-Space Projection

    The paper proposes Latent Projection for Latent Planning, noting that planning costs in latent world models are often taken as the ℓ2 distance between the predicted terminal latent and the goal latent, while the latent itself contains noise whose source is precisely the regularizer used during training to prevent collapse.

  31. arXiv cs.RO41

    Preference-PCA Pluralistic Learning (P3L): Steerable Pluralistic Policy Alignment from Anonymous Crowd Feedback

    The P3L (Preference-PCA Pluralistic Learning) framework jointly learns trajectory embeddings and a shared basis of preference directions via PCA decomposition from anonymous, heterogeneous crowd feedback, enabling personalized reward learning and downstream policy optimization without requiring annotator identities. In experiments across multiple embodied control tasks, P3L supports steerable personal alignment, generalizes beyond the offline dataset to out-of-distribution new users, and achieves competitive reinforcement learning and test performance while substantially reducing computational cost and supporting variable adaptation query budgets.

  32. arXiv cs.RO40

    Curvature as a Baseline: Dual-Color Stripe Markers Enable Monocular Pose Measurement of Cylindrical Objects

    To address the limited surface area on thin cylindrical objects that makes planar markers difficult to place, a dual-color stripe marker is proposed. It uses the cylinder's known curvature to improve monocular camera pose measurement accuracy, with the two-color boundary providing a long centerline and dots and printed edges providing features at different depths.

  33. arXiv cs.RO57

    AdaDex: Learning Generalizable Dexterous Skills from a Single Trajectory via Hybrid Dexterous Mimicking

    AdaDex is a goal-conditioned reinforcement learning framework that turns a few seconds of human hand-object trajectory (from motion capture data or HOI reconstruction from in-the-wild RGB(D) video) into generalizable dexterous skills. Its core, Hybrid Dexterous Mimicking, trains a shared policy across two sets of massively parallel environments: environments with dense motion tracking objectives inject the human prior, while environments with noised initial states and goals and a sparse object-success reward force the skill to work even when it deviates from the reference trajectory.

  39. arXiv cs.RO62

    REACT: VLM-Guided Contact Retargeting and Generative Partner Modeling for Humanoid-Human Interaction

    REACT is a framework for learning deployable humanoid-human interaction policies from human-human demonstrations, using semantic contact structure as the supervisory signal that links demonstration processing, interaction learning, and onboard control. It uses a VLM-guided contact modeling and retargeting pipeline to identify semantic contacts and transfer them onto humanoid-human geometry, then distills a privileged interaction policy into an egocentric depth policy that relies entirely on onboard sensing, and introduces a generative partner state modeling objective to predict partner motion from occluded viewpoints.

  40. arXiv cs.RO78

    SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model

    SuperMimic uses a differentiable world model of shared dynamics to train the world model and the tracking policy together in a single training loop, giving the policy analytic gradients through backpropagation-through-time over short imagined rollouts.

    Why it matters: The paper lays out a concrete path for replacing large-scale reinforcement learning training with a single differentiable world model and provides compute comparison data, making it possible to assess its impact on scaling humanoid whole-body control.