The systems in open-vocabulary mapping, side by side. Costs come from each paper and were measured on different hardware and scenes, so compare them within a row more than across rows.
| System | Language on each node | Cost reported | Setting | Assumes |
|---|---|---|---|---|
| ConceptGraphs | CLIP mean over views, plus LLaVA captions condensed by GPT-4 | 2.0 to 8.1 s per frame for the CLIP stage; 181 objects in one scene | indoor, RGB-D | a static scene, a hosted LLM for captions |
| Clio | CLIP, clustered against a task list given in advance | 0.30 s per frame on an RTX 3090; 92 objects in the same scene | indoor, RGB-D, a Spot robot | the tasks are known before the robot moves |
| HOV-SG | blended frame, crop and masked-crop CLIP, majority vote per segment | 1.5 GB over 8 scenes, against 6 GB for VLMaps | indoor, floors and rooms | a building with floors and rooms |
| Open3DSG | CLIP-space node and BLIP-space edge features, an LLM names each relation | no runtime; one 7B LLM pass per edge | indoor point clouds | a finished point cloud (batch) |
| OpenGraph | caption sentence embeddings | no runtime | outdoor driving | roads, and a path the robot has driven |
| FunGraph | GPT-4o names parts with the parent object in view | no runtime | indoor, part level | a hosted VLM |
| BiMoSG | closed-set by default, a VLM only for task-relevant objects | 0.11 s per frame, geometry stage | indoor | a GPT-4o planner over a network |
| ROMAN | CLIP per object, used to align maps between robots | under 250 KB per 40-object submap; 100 times smaller without CLIP | forest and campus, several robots | gravity direction, open-set objects |
| USS-Nav | embeds the predicted label text, never the crop | 15 Hz on a Jetson Orin NX | small drones | an LLM off board |
Notes
- Most of the compute goes into the language layer. Clio and BiMoSG get their speed by deciding early what is worth describing, and USS-Nav by embedding a word instead of an image.
- Most systems need a hosted model at some stage, so a robot that loses its link loses part of its vocabulary.
- Only ROMAN is built for more than one robot, and it uses the language features to align maps between robots.
- None of them reports what happens when two robots describe the same object differently. That is the question in 3D scene graphs and multi-robot multimodal perception.
Related: Semantic mapping, language-grounded planning.