Event
Visual Predictive Power Enables Sensorless Force-Aware Manipulation
1 reports · 1 sources
Haonan Chen et al. published a paper on arXiv proposing force-aware manipulation by visually predicting the deformation of a flexible Fin Ray gripper, requiring no force or tactile sensors at deployment. The method first uses calibration data to train a visual force estimator to annotate demonstrations with force estimates, then trains an action-force proposal policy to jointly generate candidate actions and their corresponding forces; at test time, it samples candidate actions and executes the one whose predicted force is closest to the demonstration target. The method was evaluated on four contact-intensive tasks: berry picking, grasping empty cans, in-hand reorientation, and plug insertion. The paper states that the results show visual force prediction can guide action selection at inference time without force or tactile sensors.
Report timeline
- Robot Learning with Visual Predicted Force: Achieving Force-Aware Manipulation Through Visual Force Prediction
The paper proposes achieving force-aware manipulation by visually predicting the deformation of a compliant Fin Ray gripper, without requiring force or tactile sensors at deployment. The method trains two models: first, a visual force estimator is trained on calibration data to annotate demonstrations with force estimates; second, an action-force proposal policy is trained to jointly generate candidate actions and their corresponding forces. At test time, candidate actions are sampled and the action whose predicted force is closest to a target from the demonstrations is executed. The method was evaluated on four tasks: berry picking, empty-can grasping, in-hand reorientation, and plug insertion.
arXiv cs.RO ·