Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

3 min read

Ravichandran et al. train a reinforcement learning policy that takes a 3D Dynamic Scene Graph as its observation and outputs low-level actions (move forward 0.5 m, turn 8 degrees left or right). At each step the observed part of the graph (objects, places, rooms; the mesh is dropped because it has millions of vertices) is moved into the robot’s frame, and an “Action layer” of 8 nodes on a 1 m circle around the robot is attached to nearby graph nodes, but only where the ESDF says the node and the straight line to it are in free space. Each node gets a 10-dimensional feature vector (relative position, bounding box size, node type, semantic class, a visited flag, ESDF value). A 3-round, 64-channel graph convolution network reads out the Action layer nodes into an actor-critic head trained with PPO for one million steps.

Findings

  • Multi-object search in a Unity office simulator: 7 environments, 8730 m² in total (3880 m² for training, 4850 m² for testing), 30 hidden targets per episode placed only in offices and conference rooms, 400 steps.
  • The graph policy finds 44.2% of targets and explores 59.1 m², against 39.7% and 53.3 m² for an RGB-D plus semantics policy, 38.4% for RGB alone and 31.6% for a top-down ESDF policy.
  • It collides more: 90.0 collisions per episode against 45.3 for RGB-D plus semantics. The authors put this down to inferring free space from sparse place nodes.
  • Removing rooms and objects (places only) drops targets found to 41.9% with nearly the same area explored, so the hierarchy helps the robot search the right rooms. Removing memory (current view only, no visited flag) drops it to 38.6% and the robot re-searches areas.
  • The Action layer matters most: without it targets found fall to 8.7%; without the free-space check, collisions rise from 90.0 to 229.8.
  • Noise tests on a policy trained without noise: with 0.5 m standard deviation on node positions it still finds about 40% of targets. Above roughly 40% wrong semantic labels it does worse than the places-only policy, so bad semantics actively mislead it. Adding rooms only after half their places are seen costs a little (43.0%).

Limits

  • The DSG is built offline once per scene and revealed as the robot moves, from simulator ground truth (RGB, depth, segmentation, pose). No real robot runs; the authors list integrating Hydra as future work.
  • Obstacle avoidance from graph data alone is weak, which the authors admit.
  • Semantic classes are a single integer, a closed label set; the authors suggest richer embeddings.
  • Single building, single robot, indoor offices with clear room semantics.

Relevance

It is one of the few papers that measures what a learned policy loses when the scene graph is wrong, and the semantic noise result (past a point, labels hurt more than no labels) is the kind of threshold robust robotics cares about. The visited flag is also the simplest form of the explicit memory that long-term autonomy needs. It builds on the DSG of Kimera and points to Hydra for online construction; see 3d-scene-graphs and, for language goals in place of fixed targets, HOV-SG.