Coordinating on descriptions such as “the person in the red jacket near the jetty”. Robots with different sensors (a drone’s camera, a ground robot’s lidar, a boat’s sonar) need a shared representation to agree on what they observe.
3D scene graphs are one such representation: objects, places and rooms as nodes, and their relations as edges. There’s an interactive explainer on 3D scene graphs with their history, how they’re built, and what changes outside buildings.
Questions
- What does it mean for a semantic claim from another robot to be verifiable?
- Can resilient consensus work on labels and descriptions, or does everything have to be squashed into vectors first?
- Where do multimodal foundation models help, and where do they fail?
Papers
- HEROIC (2026) runs a decentralised drone and ground robot team whose members talk only in natural language.
- SGM-SLAM matches robots’ scene graphs on object labels and centroids alone, which keeps messages small.
- Open-vocabulary mapping covers how language gets attached to map objects, and what that costs.
Related: multi-robot systems, human-robot trust, 3D scene graphs.