Event

SimRetract Achieves Zero-Shot Sim-to-Real Retractor Handover

1 reports · 1 sources

Xucheng Wang and other authors propose the SimRetract framework, which uses a policy trained only on simulated demonstrations to enable a robotic arm to take over a retractor from a surgeon during open surgery and maintain retraction during the handover, without human demonstrations or real-world fine-tuning. The authors claim this is the first zero-shot sim-to-real transfer for a surgeon-robot collaborative task in open surgery. In 50 model demonstrations set up by surgeons, the policy successfully grasped the retractor in 90% of cases and maintained retraction during handover in 78%. The authors state that results on unseen retractors suggest the policy may generalize to surgical instruments with similar grasping geometries. On the real robot, merely removing the instrument mask reduced performance to 22% grasping and 12% maintaining.