Event
AOV-GS: An Active Semantic Mapping System Guided by Foundation Models
1 reports · 1 sources
Rodion Shkokov, Tatiana Zemskova, and Dmitry Yudin have released AOV-GS, an active semantic mapping system based on 3D Gaussian splatting that requires no human-annotated masks or closed semantic categories, uses frozen foundation models to guide exploration, and has released the code. In the coarse exploration stage, the system reranks geometric candidate viewpoints by the novelty of CLIP mask embeddings obtained from SAM instance segmentation, biasing exploration toward regions with high semantic information content; a lightweight scene language field fitted offline directly aligns with the collected CLIP embeddings, supporting open-vocabulary text-query segmentation. In comparisons using the same SAM-CLIP perception stack, the authors report that AOV-GS's language field achieves an average mIoU 1.3 percentage points higher than an active strategy that plans only by geometry on Replica (8 scenes, 34.6%) and 1.8 percentage points higher on Matterport3D (5 scenes, 23.1%); novel-view PSNR is comparable to the geometric baseline, and depth L1 improves on Replica.
Report timeline
- AOV-GS: Active Open-Vocabulary Semantic Mapping with 3D Gaussian Splatting
AOV-GS is an active 3D Gaussian Splatting mapping system that uses frozen foundation models to guide exploration, without requiring manually annotated segmentation masks or closed semantic categories. In its coarse exploration stage, it re-ranks geometric candidate views by the novelty of CLIP mask embeddings computed from SAM instance segmentation, biasing exploration toward semantically informative regions; a lightweight scene language field fitted offline directly aligns the collected CLIP embeddings, enabling open-vocabulary text-query segmentation.
arXiv cs.RO ·