# CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping and LLM Agent for Immersive Collaborative VR Teleoperation

- Source: arXiv cs.RO
- Published: 2026-10-27T04:00:00.000Z
- RoboHot: https://robohot.news/en/items/djot3swt9v1dnk93u7t3rtz95
- Original: https://arxiv.org/abs/2609.31418

## Summary

CognitiveReality builds a semantically indexed Gaussian-TSDF map from a robot's RGB-D stream, shared by a VR operator and a tool-calling language agent. A single mapping program adapts to different platforms through configuration alone, accepting poses from robot SLAM, joint kinematics, motion capture, or visual tracking, and handles localization outages with shadow tracking and keyframe-anchored PnP, maintaining open-vocabulary instance identities and per-object quality at 2 Hz.