Event

FlowHOI: Flow Matching for Generating Hand-Object Interaction Sequences

1 reports · 1 sources

Huajian Zeng and other authors propose FlowHOI, a two-stage flow matching framework that generates hand-object interaction sequences containing hand pose, object pose, and contact states, conditioned on first-person observations, language instructions, and 3D Gaussian splatting scene reconstruction. The authors claim that on the GRAB and HOT3D benchmarks, FlowHOI achieves the highest action recognition accuracy, with a physical simulation success rate 1.7 times higher than the strongest diffusion model baseline and a 40 times speedup in inference. The authors also claim to have demonstrated real-robot execution on four dexterous manipulation tasks, indicating that transferring the generated hand-object interaction representation to a real robot execution pipeline is feasible.