Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environments

3 min read

Khronos builds a dense metric-semantic map of a scene at every past time, online, so a robot can answer “what was here at 1:52?” as well as “what is here now?”. The authors call the problem Spatio-temporal Metric-semantic SLAM (SMS) and factor it with one assumption: over a short window, both odometry drift and scene change are small. Observations are grouped into object fragments inside an active window, where short-term motion (people, a pushed cart) is tracked with a free-space motion cue. A global stage adds fragment poses to Hydra’s deformation graph and solves it with truncated least squares and GNC, treating same-label, overlapping fragments as candidate associations. A last stage decides when each object appeared or vanished by checking it against a library of camera rays.

Findings

  • The ray check separates evidence of absence from absence of evidence. If a ray to a background vertex passed through where an object was, the object was gone at that time; a timestamp counts as absence only if at least 60% of rays in a 5 s window agree.
  • Two TESSE simulations: Apartment (87 s, 39 m, 64 static objects, 10 dynamic, 6 changes) and Office (217 s, 181 m, 196 objects, 6 dynamic, 8 changes).
  • Office with Kimera visual-inertial poses: change detection F1 62.0 against 38.6 for Panoptic Mapping, dynamic-object F1 53.9 against 23.5 for Dynablox, and static-object F1 73.1 against 62.1 for Hydra.
  • Apartment with ground-truth poses: object recall 83.9 against Hydra’s 28.0. Hydra segments objects out of the background mesh and misses small items.
  • Swapping ground-truth semantics for an open-set SAM plus CLIP frontend barely changes Apartment change F1 (64.4 against 64.6).
  • The active window takes 45.5 ± 9.2 ms per frame (22.2 FPS) on a laptop i7-12700H CPU. Optimisation after a loop closure can take several seconds; change inference stays under 100 ms.
  • On a Jackal, against manual labels: object precision 86.9 and recall 97.7, change precision 87.0 and recall 66.7. The table prints a change F1 of 93.9, which cannot follow from that precision and recall, so I quote only the pair. It also ran on a Spot across a building floor (qualitative).

Limits

  • Fragment associations use bounding-box centroids as translation-only constraints, so partial views and occlusion hurt them. No 6D registration (authors).
  • Association is purely geometric, so a moved object becomes one disappearance plus one appearance. Its history is not linked (authors).
  • The ray test needs a reference surface behind the object. The authors expect it to fail in open spaces with sparse surfaces, which covers most outdoor, aerial and open-water scenes.
  • Every fragment stays in memory, so the problem grows without bound (authors).
  • Motion detection misses people walking straight towards or away from the camera, because the free space behind them is never observed (authors).
  • Single robot. The main benchmark uses simulator semantics.
  • SuperMap reports that Khronos could not keep up in its tests because of slow semantic inference. Khronos’s own timing assumes segmentation is already there, so the two claims probably differ on what gets counted.

Relevance

Khronos defines what “the map at time t” should mean, and its fragment and ray machinery is reusable for long-term autonomy and dynamic scenes. It also inherits the static-world baseline of Hydra and the short- and long-term split of 3D VSG.