Enhancing Situational Awareness in Underwater Robotics with Multi-modal Spatial Perception

3 min read

Kaveti et al. collect field data from Minerva II, NTNU’s 2.5-tonne work-class ROV, in the Trondheim Fjord, and try three perception pipelines on it. The ROV’s own cameras sit on pan-tilt units that pilots move during dives, so they built a rigid prototype rig of three hardware-synchronised DWE StellarHD cameras with a STIM300 IMU and a Khadas VIM4 single-board computer. The pipelines are MCSLAM, a multi-camera SLAM system using a generalised camera model, with IMU and DVL factors added to its back end; DROID-SLAM, which uses learned optical flow and dense bundle adjustment; and a DeepLabv3 model labelling pipeline, pipeline support and background, projected onto the DROID-SLAM point cloud. It is a workshop paper and the results are qualitative.

Findings

  • Datasets: the Hercules shipwreck (a 285 s run and a 4,260 s, roughly 71 minute, survey in concentric circles), the PLM subsea module (1,072 s, with a detour over featureless seafloor and back), the TBS seafloor with OAK-D stereo in turbid water (537 s), a pipeline followed from 70 m up to 30 m depth (736 s), and a 71 s synthetic wreck in Gazebo.
  • Multi-camera SLAM with IMU and DVL keeps tracking and closes loops on the wreck, the PLM module and the TBS seafloor loop. Without ground truth, the evidence is continuity of tracking and the shape of the point clouds.
  • DROID-SLAM produces a dense reconstruction of the Hercules wreck, but each circular pass drifts until global bundle adjustment runs. It has no loop closure or relocalisation and loses track on the PLM dataset when the ROV leaves the module and returns.
  • The semantic point cloud on the pipeline dataset looks clean, which the authors attribute to good visibility and to training the segmenter on nearby, similar pipelines.

Limits

  • No trajectory or map error numbers, and no runtime or compute figures.
  • The ROV carries a Norbit WBMS forward-looking sonar, but none of the results shown use it. The acoustic sensor in the pipelines is the DVL.
  • Segmentation is closed-set with three classes. The authors expect failures under poor lighting and when thrusters stir up sediment, “where no amount of image processing would help”, and they note how little underwater training data exists.
  • Calibration breaks when operators change zoom, pan and tilt during a dive; refraction through housings adds to it. They name online recalibration as an open problem.
  • DROID-SLAM takes one camera and fuses no inertial or acoustic data, so drift compounds whenever the view goes blank.

Relevance

The list of field failures (moving cameras, stirred sediment, a blank water column) is a useful checklist for robust robotics and for long-term autonomy in the water; see underwater perception.