Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction

1 min read

This paper adapts vision foundation models trained on RGB to imaging sonar, since no large free sonar datasets exist to train an acoustic foundation model from scratch. It uses the geometric relationship between camera and sonar projection, plus physics-based noise models, to generate synthetic training data. The adapted models allow what the authors call the first experimental sonar-based pose-free 3D reconstruction.

Relevance

If RGB foundation models can be bent to acoustics, underwater robots get some of the 3D reconstruction tools land robots already have. Read for underwater perception and foundation models for robotics in multi-domain perception.