Event

AFFORD2ACT: Sparse 2D Keypoints Drive Robot Manipulation Policies

1 reports · 1 sources

Anukriti Singh et al. propose the AFFORD2ACT method, which uses a language-conditioned affordance model to localize task-relevant regions, selects at most 15 object keypoints plus 4 end-effector keypoints, compresses the policy input to 19 2D keypoints (38 coordinates), and then uses a lightweight gating mechanism to decide which keypoints to attend to during execution. On six real robot manipulation tasks, the method outperforms the evaluated RGB, RGB-D, and dense-affordance behavior cloning baselines, achieving an average success rate of 82% in held-out scenes, and its performance degrades less when the number of demonstrations is reduced. Replacing the learned gating with uniform weights drops the success rate from 92% to 52%. The authors note that this representation relies on reliable affordance localization and keypoint tracking, and remains sensitive to incorrect masks, occlusion, tracking drift, and distractors.