Collaborative Dynamic 3D Scene Graphs for Automated Driving

3 min read

CURB-SG builds a dynamic 3D scene graph of a city from several cars with panoptic lidar. Each car runs lidar odometry (FAST-GICP) on the static points and sends keyframes, each a pose plus a labelled static point cloud, to a central server, along with the points of other vehicles it sees. The server extends HDL Graph SLAM to many agents: it finds intra- and inter-agent loop closures by searching a radius around each new keyframe and accepting ICP matches below a fitness threshold, optimises the pose graph with g2o, and contracts redundant edges so the graph stops growing. From the agents’ own paths and the tracks of observed cars it builds a lane graph, splits the city into intersections and roads, and stacks six layers: city, intersections and roads, static landmarks, dynamic vehicles, lane graph, and the pose graph with its point clouds.

Findings

  • Everything runs in CARLA (towns 01, 02, 07 and 10), ten runs each, with the simulator’s panoptic labels and consistent instance IDs. The authors cite a lack of real multi-agent lidar datasets.
  • More agents help. In town01, keyframe RMSE drops from 0.735 m with one agent to 0.132 m with three, street-sign RMSE from 0.865 m to 0.169 m, and the time to map 85% of the town from 2502.5 to 1134.4 simulation steps.
  • Gaussian noise of N(0, 0.02) on the lidar scans barely moves the errors (town01, three agents: 0.159 m against 0.132 m).
  • Adding observed vehicles to the lane graph raises single-agent graph IoU in town02 from 0.386 to 0.690 and TOPO recall from 0.281 to 0.562, while TOPO precision drops from 0.810 to 0.678 because oncoming cars’ centroids are biased towards the centre line.
  • Intersection detection from the lane graph is at least 20% more precise than a skeletonisation baseline on the road surface.
  • Edge contraction keeps the count of optimisable pose-graph nodes bounded while keyframes keep arriving.

Limits

  • A new agent is registered to the global graph with its ground-truth absolute pose as the initial guess, because the radius-based loop-closure search needs one. The authors suggest GNSS or a global descriptor search would replace this, and list known initial poses as a limitation.
  • Loop closures pass on an ICP fitness threshold. No outlier-robust backend is described, so one bad inter-agent match goes straight into the graph.
  • Ground-truth panoptic labels from the simulator, which the authors also list as a limitation.
  • One central server, lossless links, identical sensors and segmentation on every car. A decentralised real-time version is future work. Keyframing is motivated by bandwidth, but no bandwidth figures are reported.
  • Contraction keeps the newer point data and drops the older, which keeps the map current but throws away history. The dynamic layer holds vehicles seen now; it is not a record of change.
  • No pedestrians yet.

Relevance

CURB-SG is the outdoor sibling of Hydra-Multi, and its shortcuts (ground-truth initial poses, simulator labels, a trusted server, no outlier rejection) are the first things a real fleet without perfect GPS would trip over. It feeds collaborative SLAM, dynamic scenes and multi-robot multimodal perception.