Event
CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping and VR Teleoperation
1 reports · 1 sources
Timofei Kozlov and other authors released a paper proposing the CognitiveReality framework: it builds the robot's RGB-D stream in real time into a semantically indexed Gaussian-TSDF map, shared by VR operators and a language agent that can invoke tools. The authors claim that the same mapping program can adapt to different platforms through configuration alone, and can interface with robot SLAM, joint kinematics, motion capture, or visual tracking poses, and that it handles localization interruptions with shadow tracking and keyframe-anchored PnP. The authors report that the framework was deployed on two quadruped robots in real-world settings; the agent executed 26 of 30 navigation requests and 20 of 20 re-observation requests, with object quality improved by 2–5 dB; during SLAM interruptions of 5–40 seconds, pose error remained within 1–8 cm. These results are all as stated by the paper's authors and await independent verification.
Report timeline
- CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping and LLM Agent for Immersive Collaborative VR Teleoperation
CognitiveReality builds a semantically indexed Gaussian-TSDF map from a robot's RGB-D stream, shared by a VR operator and a tool-calling language agent. A single mapping program adapts to different platforms through configuration alone, accepting poses from robot SLAM, joint kinematics, motion capture, or visual tracking, and handles localization outages with shadow tracking and keyframe-anchored PnP, maintaining open-vocabulary instance identities and per-object quality at 2 Hz.
arXiv cs.RO ·