A 3D scene graph stores a map as a graph. Nodes are things (objects, places, rooms, floors, buildings, people), edges say how they relate (on, in, next to, part of), and every node points at the patch of the metric map it occupies. A robot can then answer “where is the fire extinguisher nearest the stairs” without searching a point cloud, and an LLM can read the map as text.
The interactive explainer on 3D scene graphs has the history and small simulations. This note is the reading map that goes with it. The best single entry point is the 2026 survey by Rotondi and others, written by people from most of the groups that built the field.
History
| Years | What changed | Papers |
|---|---|---|
| 2019 to 2020 | Offline, static, closed set: the hierarchy and the first datasets | Kim, Armeni, 3DSSG, 3D DSG |
| 2021 to 2024 | Built online, inside SLAM, with a backend that can undo bad loop closures | Kimera, Hydra, S-Graphs, Hydra (IJRR) |
| 2023 to 2025 | Open vocabulary: CLIP features and VLM captions on nodes, LLMs as the reader | ConceptGraphs, Clio, HOV-SG, Open3DSG, FunGraph |
| 2024 to 2026 | Change over time, object shape, physics, cheaper sensors | Khronos, SuperMap, Hydra++, Hickory, Picasso, vS-Graphs, Mono-Hydra |
| Teams | One graph from several robots | Hydra-Multi, CURB-SG, ROMAN |
Vision and robotics
The survey splits the field in two. The vision camp predicts a graph from a finished, fused point cloud, learns the relations, and reports triplet recall. The robotics camp builds the graph incrementally from a live sensor stream, fights pose drift, and reports trajectory error. Methods from one camp are rarely tested on the other’s benchmarks, and most robotics systems are evaluated on their own data, so they are rarely compared head to head.
Within the robotics camp there are two designs. The Hydra line builds the graph beside the SLAM backend and corrects every layer at once through a deformation graph, with graduated non-convexity to reject wrong loop closures (see robust estimation). The S-Graphs line puts walls, rooms and floors inside the factor graph, so the structure constrains the pose. Without its room layer, S-Graphs’ trajectory error grows to 0.40 m and 1.01 m in the larger simulated scenes.
Multi-robot construction
Only Hydra-Multi and CURB-SG build one graph from several robots. Both quietly assume four things:
- A shared frame. Hydra-Multi finds it from inter-robot loop closures and survives 80% to 90% outlier closures in look-alike dorm rooms. CURB-SG starts every car at its ground-truth pose. Tunnels, mines and multi-storey car parks give neither GPS nor distinctive places.
- Honest teammates. Outlier rejection handles perceptual aliasing, but every robot’s own graph and odometry are taken as true. Hydra-Multi undoes all proposed merges if fewer than half survive, so one robot that is consistently wrong could cancel the good merges from the rest.
- Good links. Both are centralised. Hydra-Multi resends each robot’s whole graph and mesh at every update, which its authors call naive. Neither tests dropped or delayed messages.
- The same labels. Hydra-Multi merges two objects only when their labels match. Free-text captions from two robots would almost never match.
The 3DSSG paper measures this: graph-based scene retrieval gets the right scene first 96% of the time from ground-truth graphs and 34% from predicted ones. Matching two robots’ graphs is only as good as each robot’s perception. The survey gives multi-robot construction one sentence and never mentions trust or communication. This is the question the last section of the explainer asks: given several graphs that disagree, which claims should the merged map keep? It links resilient consensus to perception.
Outdoors and in the air
Almost every system assumes a building. Outdoors there are no rooms: OpenGraph builds a lane graph from the path the robot drove, and the Strader ontology paper gets regions such as beach, road and field from an LLM-written ontology using 0.1% of the labels. On small drones, USS-Nav runs at 15 Hz on a Jetson Orin NX by embedding label text instead of image crops. More on small aerial robots in aerial swarms.
Numbers
- Hydra on a Jetson Xavier NX CPU: about 75 ms for objects, 33 ms for places, 55 ms for rooms (from a 2025 talk by Strader).
- Clio keeps 92 objects where ConceptGraphs keeps 181, at 0.30 s per frame against 2.0 s.
- ROMAN: a 40-object submap is under 250 KB, and a kilometre of map under 25 MB.
- Khronos calls an object gone when 60% of camera rays in a 5 s window pass through where it was.
- Hydra++ raises object precision from 0.670 to 0.787 by checking each learned mesh against its mask.
Questions
- How do robots agree on a node when one says “chair”, another says “office chair, grey”, and a third only has a lidar cluster with no label?
- What is the smallest message that still lets a teammate query my graph in language?
- Which checks can a robot run on a teammate’s graph before merging it: physical plausibility (Picasso), consistency with its own observations, agreement with a third robot?
- Khronos’s absence test needs a wall behind the object. What counts as evidence that something is gone in an open field?
- How should a graph forget, over months?
Related: open-vocabulary mapping, language-grounded planning, dynamic scenes, collaborative SLAM, semantic coordination, multi-robot multimodal perception.