Event
SenseTime and Partner Team Evaluate GPT-6-Astra Embodied Capabilities and Open-Source Evaluation Framework
5 reports · 5 sources
SenseTime SenseNova, together with Daxiao Robotics, Ropedia, Nanyang Technological University Singapore, the University of Tokyo, and the Hong Kong University of Science and Technology (Guangzhou), released a technical report evaluating GPT-6-Astra's capabilities in robotic manipulation and navigation, and simultaneously open-sourced the SenseNova-RoboRSI evaluation framework and a 27B embodied model. Under a unified protocol (allowing in-context learning ICL from training-set examples and using at most 10x the number of action steps), the team evaluated on 12 representative benchmarks covering tabletop manipulation, humanoids, mobile platforms, and dexterous hands, achieving what the paper calls unprecedented performance. Experiments on 8 real robot configurations show a clear gap between simulation and real machines: fine manipulation on real robots fails frequently, and the robot demonstration data required for ICL is costly to collect, making it difficult under physical constraints to trade longer execution time for higher success rates. The real-robot experiments used a zero-shot setting, providing no demonstrations or knowledge packs, with budgets measured in wall-clock time (typically 60 minutes), and success judged by an operator based on physical results. The team also developed a prototype that derives tasks from real needs, provides no in-domain demonstrations, and judges the agent with a continuously running world clock. The report concludes that GPT-6-Astra has largely crossed the threshold of embodied competence but has not yet reached practical embodied intelligence. The first batch of code in the open-source repository includes the Agent runtime, benchmark evaluation, and LIBERO series adapters; model checkpoints, training code, and historical ICL packs are not included in the release.
Report timeline
- SenseTime and joint team evaluate GPT-6-Astra embodied capabilities, open-source SenseNova-RoboRSI framework and 27B model
A joint team including SenseTime and The Hong Kong University of Science and Technology (Guangzhou) released the report "Has GPT-6-Astra Achieved Embodied Intelligence?", systematically evaluating GPT-6-Astra's embodied capabilities across 12 benchmarks, and simultaneously open-sourced the SenseNova-RoboRSI evaluation framework and a 27B embodied model.
3D视觉工坊 · 微信公众号 ·
- Edge AI Daily Morning Brief (October 21)
An evaluation by SenseTime together with teams including Daxiao Robotics and Nanyang Technological University shows that GPT-6-Astra came close to full marks on several of 12 simulation benchmarks (100% on LIBERO-Long++, 98.8% on SIMPLE), but after deployment on 8 real robot platforms, all 11 attempts to plug in a plug failed and folding clothes ran into trouble. Its high scores also depend on two "boosters": in-context learning (ICL) and relaxing the action step limit to 10x; on the long-horizon household benchmark BEHAVIOR, its success rate was only 37.0%.
钛媒体 ·
- SenseTime SenseNova-RoboRSI unified Harness raises GPT-6-Astra success rate on RoboDojo from 15.83% to 54.17%
A joint research team from SenseTime, Nanyang Technological University and other institutions has introduced the SenseNova-RoboRSI unified Harness, which raises GPT-6-Astra's manipulation success rate on 42 RoboDojo tasks from 15.83% in the baseline configuration to 54.17%, with model parameters unchanged.
PaperWeekly · 微信公众号 ·
- SenseTime and partner teams evaluate GPT-6-Astra's embodied capabilities: near-perfect scores in simulation, frequent failures in fine manipulation on real robots
SenseTime SenseNova, together with Daxiao Robotics, Ropedia, Nanyang Technological University Singapore, and the University of Tokyo, released "Has GPT-6-Astra Achieved Embodied Intelligence?"
机器之心 · 微信公众号 ·
- Has GPT-6-Astra Achieved Embodied Intelligence? SenseNova Joint Team Evaluation Report
The SenseNova joint research team released a report evaluating GPT-6-Astra's embodied capabilities in robotic manipulation and navigation. Under a unified protocol, the team evaluated it on 12 representative benchmarks spanning tabletop manipulation, humanoids, mobile platforms, and dexterous hands, allowing in-context learning (ICL) from training-set demonstrations and substantially increasing the number of execution steps, achieving what the paper calls unprecedented performance.
arXiv cs.RO ·