Event
NVIDIA Open-Sources GATOR: Reconstructing Posed 3D Objects from Casual Photos
2 reports · 2 sources
NVIDIA and Simon Fraser University proposed and open-sourced GATOR, which can reconstruct complete 3D objects that are aligned with the scene and have pose and texture from casually taken photos that require neither object isolation nor calibration. The method first uses generative reconstruction to obtain complete assets with scene poses, and then uses a multimodal agent for local refinement; it achieves 96.36% ADD-SB@0.1 on HANDAL and 95.63% on ScanNet++. GATOR is trained only on synthetic scenes (about 980,000 object assets plus a structured indoor dataset), and is used on real desktop and indoor scans with zero real-machine fine-tuning. Reports say that without refinement it takes about 0.5 minutes to outperform GPT-6-Astra running for 20 minutes, but these metrics are computed on standard datasets with matched cameras and lighting, and are not equivalent to absolute quality from arbitrary viewpoints in real scenes. The method relies on target masks and camera depth estimated from images. The authors note that dependence on synthetic data, binding to commercial APIs, and stability under extreme poses remain open problems for deployment.
Report timeline
- NVIDIA and Simon Fraser University propose GATOR: reconstructing posed 3D assets from casual photos
A NVIDIA team, together with Simon Fraser University, proposes GATOR, which uses generative reconstruction as a base and multimodal agents for local refinement to reconstruct complete 3D objects with scene poses from casually taken photos. The method is trained only on synthetic scenes and, without any real-world fine-tuning, leads baselines on geometry, appearance, and pose metrics in real tabletop and indoor scans; without refinement, it takes about 0.5 minutes to outperform GPT-6-Astra running for 20 minutes. The authors also note that reliance on synthetic data, binding to commercial APIs, and stability under extreme poses remain open issues for deployment.
深蓝AI · 微信公众号 ·
- NVIDIA open-sources GATOR, reconstructing complete 3D objects with poses from casual photos, pose accuracy 96.36%
NVIDIA and Simon Fraser University propose and open-source GATOR, which reconstructs complete 3D objects aligned with the scene and with poses from casual photos that require no object isolation or calibration, achieving 96.36% ADD-SB@0.1 on HANDAL and 95.63% on ScanNet++.
3D视觉工坊 · 微信公众号 ·