LOST-3DSG: Lightweight Open-Vocabulary 3D Scene Graphs with Semantic Tracking in Dynamic Environments

2 min read

LOST-3DSG keeps an open-vocabulary 3D scene graph up to date while objects move, without storing dense CLIP features. A VLM detects objects and extracts attributes such as colour, material and shape. Those attributes become cheap word2vec and sentence embeddings, which are used to re-identify objects across frames. Per the README, the system is two ROS 2 nodes: a perception module that produces 3D boxes, point clouds and descriptions, and a scene updater that associates detections and sorts them into new objects, moved objects and objects that have left the view. I have read only the abstract and the README.

Findings

  • The authors report better tracking than approaches built on high-dimensional visual embeddings, in qualitative and quantitative tests in a real environment with a TIAGo robot. The abstract gives no numbers.
  • The authors present the work as foundational insights and say it is incomplete.

Limits

  • Abstract-level reading only.
  • My own reading: text embeddings of VLM-described attributes inherit the VLM’s mistakes, and two identical chairs get identical descriptions, so re-identification has to lean on geometry.
  • Single robot, indoor, RGB-D.

Relevance

LOST-3DSG is a lightweight take on dynamic scene tracking in open-vocabulary maps, cheaper than the identity machinery in SuperMap and Khronos. Short text attributes are also far cheaper to send between robots than feature vectors, which helps multi-robot perception over thin links.