Event

PolyUMI: An Open-Source Multimodal Demonstration Collection Platform and the VisTA Policy

1 reports · 1 sources

Researchers from Cornell University and other institutions (Conor W. Hayes et al.) have released the open-source platform PolyUMI and the multimodal policy VisTA for collecting robot demonstration data and deploying policies. PolyUMI's lightweight wireless handheld gripper can synchronously record wrist camera, optical tactile, contact audio, and proprioceptive observations without a wired workstation; the same sensing finger can be transferred to the robot end effector, maintaining consistent sensing geometry between the collection and execution stages. The researchers report that in object reasoning, slip control, and contact-rich manipulation experiments, tactile and audio provided task-relevant information beyond vision, and VisTA performed comparably to or better than existing multimodal policies.