Skip to content

Event

OpenFlyV2 releases paper on onboard vision-language-action model

2 reports · 2 sources

The OpenFlyV2 author team released a paper proposing a unified onboard vision-language-action architecture that uses a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous flight control within a single model. According to the paper, the model generates action chunks fully onboard at 10 Hz on an NVIDIA Jetson Orin NX, with each chunk containing 12 waypoints; it transfers zero-shot from simulation to a real quadrotor, achieving a 48% navigation success rate in four real environments, without using real aerial training data. The team also built a simulation dataset of approximately 70,000 navigation trajectories and 60,000 vision-language instruction-response pairs, and stated that the code will be made public on GitHub.

Report timeline

  1. OpenFlyV2: An onboard vision-language-action model for aerial perception and navigation

    OpenFlyV2 is a unified onboard vision-language-action architecture that uses a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous flight control within a single model. The authors built a simulation dataset of approximately 70,000 navigation trajectories and 60,000 vision-language instruction-response pairs. On an NVIDIA Jetson Orin NX, the model runs fully onboard and generates action chunks of 12 waypoints each at 10 Hz.

    arXiv cs.RO ·

  2. OpenFlyV2 releases onboard vision-language-action model for aerial perception and navigation

    OpenFlyV2 is a unified onboard vision-language-action architecture that uses a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous flight control within a single model. The model generates action chunks of 12 waypoints each entirely onboard at 10 Hz on an NVIDIA Jetson Orin NX, and transfers zero-shot from simulation to a real quadrotor, achieving a 48% navigation success rate in four real environments without using real aerial training data.

    HuggingFace Daily Papers ·

Related topics

Perception and localizationPlanning and decision makingRobot models and learningData and simulationResearch papers

Interest trend

Current
3
Peak
4Nov 4 01:00
Past 24 hours
–

The trend compares participants observed throughout the same period, so its range may be smaller than the current interest count. Move or tap to inspect each hour; use the arrow keys on a keyboard.