Event

KAIST proposes robot-centric pointmap VLA input method

1 reports · 1 sources

KAIST AI and Holiday Robotics propose a "robot-centric pointmap" as input to vision-language-action (VLA) models: RGB-D observations are transformed into the robot coordinate frame, the H×W grid is preserved, and the result is added element-wise to RGB tokens to inject into a pretrained VLA. The team reports that on RoboCasa, π0.5 improves by 7.6 points. The method requires calibrated camera intrinsics and extrinsics at both training and test time, so it applies only to settings where calibration is available; camera-change experiments cover only camera position and extrinsic changes, not the number of cameras or field of view. The comparison with point clouds used only a single sampling budget, and the authors say a larger budget could narrow the reported gap; the injection method of the pointmap relative to the action expert and its interaction with the pretraining recipe were also not ablated, so the best combination remains undetermined.