Event

FoveaVLA Replaces Wrist Cameras with Gaze Actions, Surpassing Baseline Success Rate

1 reports · 1 sources

Bingao Chen and other researchers propose the FoveaVLA model, which replaces wrist cameras in VLA with learned gaze actions: while predicting action chunks, it also predicts the interaction region to be observed in the next policy query, crops that region from the original-resolution image, and feeds it into the policy together with the panoramic view. The model is built on π₀.₅ in two configurations, Single and Dual, using one to two external stereo cameras to provide panoramic and foveal RGB and depth inputs, and is post-trained on DROID. In evaluations on four sets of real tasks, FoveaVLA-Single achieves a 70.8% success rate using one external RGB-D view, and FoveaVLA-Dual achieves 77.1%, while the π₀.₅ baseline with wrist cameras achieves 52.1%; both match the baseline on ordinary pick-and-place tasks and improve success rates on tasks with distractors, cloth manipulation, and distant objects.