Hydra-Multi: Collaborative Online Construction of 3D Scene Graphs with Multi-Robot Teams

4 min read

Hydra-Multi builds one 3D scene graph from a team of robots, online. Each robot runs its own copy of Hydra and periodically sends its whole local scene graph to a central control station. The station searches for inter-robot loop closures with Hydra’s hierarchical descriptors (places, then objects, then visual appearance), checks each match geometrically with RANSAC or TEASER++, estimates every robot’s frame by robust pose averaging, and optimises the joint graph as an embedded deformation graph with Graduated Non-Convexity (GNC). Places and objects that two robots both mapped are proposed as merges, and a merge is kept only if GNC classes it as an inlier.

Findings

  • The robots need no initial pose calibration. Pairwise frame offsets come from a truncated least squares average over all loop closures between two robots, chained along a spanning tree of the robot graph. A robot counts as initialised once it has at least 5 inlier loop closures.
  • On the Simmons dormitory data, identical rooms with identical furniture make the frontend return 80% to 90% outlier loop closures, and the frames still align.
  • Trajectory error (ATE): 0.25 m on simulated uHumans2 against 0.59 m for a centralised Kimera-Multi, and 3.92 m against 4.99 m on the multi-floor SidPac recordings. On Simmons the lidar-based LAMP 2.0 stays ahead (0.73 m and 0.58 m against 0.99 m and 0.79 m).
  • The ablation shows where the accuracy comes from. On Simmons SM1, objects found and objects correct fall from 59.3% and 55.2% to 23.8% and 16.2% without initial alignment, and to 37.3% and 29.5% without merge proposals.
  • Two Jackals mapped a full dormitory floor in about 30 minutes. One robot covering similar paths took about 50.
  • A heterogeneous run pairs a camera robot with a lidar robot that has no semantics. The lidar robot still contributes mesh and places; only the camera robot fills the object layer. Any robot whose map matches at least one layer can join.
  • The frontend stays near 100 ms per iteration. The backend grows with graph size, so loop-closure corrections lag behind.

Limits

  • One control station receives everything and does all the fusion. The authors list a distributed backend as future work.
  • Each robot resends its entire scene graph and mesh at every update, and those dominate bandwidth (loop-closure descriptors and deformation-graph inputs stay under 1 MB). The authors call this naive.
  • The world is assumed static. The authors list dynamic objects as future work.
  • Peers are trusted. GNC rejects wrong loop closures and wrong merges caused by perceptual aliasing, but nothing models a robot that sends a corrupted or fabricated graph, or one whose odometry has quietly failed. Reconciliation is all or nothing: if fewer than half of proposed merges survive, every merge is undone.
  • Object merges require the same semantic label and overlapping boxes, so robots must share a label set. Two segmentation networks are used across datasets (HRNet, OneFormer), never mixed within one team.
  • The frame alignment is computed once and recomputed only if it disagrees with the optimised solution by more than 10 m.
  • Links are assumed reliable. There is no test with dropped, delayed or rate-limited messages, and the two SidPac “robots” are two hand-held recordings.

Relevance

Hydra-Multi shows how far outlier-robust estimation goes for multi-robot scene graphs: GNC copes with aliasing, yet every robot’s local graph is taken as honest. A team whose robots see different things, or where one robot’s map is quietly wrong, needs more than this; see collaborative SLAM, outlier-robust estimation, Byzantine fault tolerance and multi-robot multimodal perception. The outdoor counterpart is CURB-SG.