ROMAN (Robust Object Map Alignment Anywhere) relocalises a robot against its own old map or another robot’s map by aligning small maps of open-set objects. Each RGB-D frame is segmented with FastSAM, each segment gets a CLIP embedding, and segments are tracked into 3D objects. Once the robot moves on, each object shrinks to a centroid, a four-number shape descriptor and a mean 768-dimensional CLIP vector. Submaps of these objects are matched with a CLIPPER-style graph-theoretic data association that folds shape similarity, semantic similarity and the IMU’s gravity direction into the consistency scores, instead of filtering bad matches before or after.
Findings
- As a loop closure module in multi-robot SLAM on the Kimera-Multi data, the abstract reports a 35% lower trajectory error than visual-feature loop closures on the hardest sequences, and the intro reports 8% lower error across the full 6 to 8 robot problems.
- The authors report up to 45% better pose estimation in opposite-direction relocalisation, where image-based methods tend to fail.
- A submap holds at most 40 objects, so a packet is under 250 KB, and a 1 km trajectory fits in under 25 MB. Dropping the CLIP vectors cuts map size by a factor of 100 for a 12% drop in alignment success.
- Tested in urban (MIT campus), off-road forest and a small indoor ground-to-aerial cross-view setup with 20 traverse pairs.
Limits
- The authors admit that inconsistent segmentation still produces duplicates (a car and each of its doors as separate objects), and that ground, walls and buildings are thrown away.
- Registration uses centroids only. They suggest coarse-to-fine alignment as future work.
- Tested up to eight trajectories of about 1 km; longer missions mean many more submaps to register.
- I have only read the abstract, intro and parts of the arXiv HTML, so the full ablations are unchecked.
Relevance
Here open-set semantics does a geometric job: CLIP vectors disambiguate matches between robots that saw a place from opposite sides. The 100 times size cut from dropping CLIP is a useful number for any team with thin links; see collaborative SLAM, open-vocabulary mapping and multi-robot multimodal perception. Compare the outdoor two-robot system in Strader et al. 2025 that uses ROMAN for relocalisation.