Event

HKU Open-Sources GenSfM: Joint Reconstruction of Objects and Poses from Four Views

1 reports · 1 sources

A team at the University of Hong Kong has open-sourced GenSfM, which uses a native 3D diffusion model to simultaneously denoise object shape and per-view camera poses within a single multimodal sequence, requiring only 2 to 4 uncalibrated images and no camera intrinsics or extrinsics. The team reports that on Toys4k and GSO, its novel-view PSNR is 3.2–4.5 dB higher than the multi-view baseline TRELLIS-M, its median Chamfer distance is reduced by about 64%–74%, and its pose Acc@5° exceeds 96%. The paper has been accepted to NeurIPS 2026, and the code and models are publicly available. When using one's own images, 2–4 views of the same rigid object must be prepared (four views covering different sides are recommended), along with RGBA foreground masks with transparent backgrounds; the first run requires downloading about 5.8 GB of pretrained models and the DINOv2 encoder from Hugging Face. The trained UV encoder required for UV target encoding is not included in the inference bundle. Results are limited to object-centric inputs; extending to more views, scene-level inputs, and real captures with looser segmentation is listed by the team as future work.