Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching

2 min read

Fast-FoundationStereo shrinks the FoundationStereo model until it runs in real time while keeping most of its zero-shot accuracy. Three steps do the work: knowledge distillation compresses the hybrid backbone into one small student (an EdgeNeXt network, per the model card), a blockwise architecture search picks cost-filtering blocks under a latency budget, and structured pruning trims the iterative refinement stage. An automatic pseudo-labelling pipeline curates 1.4 million in-the-wild stereo pairs to add to the synthetic training data.

Findings

  • The abstract reports over 10x the speed of FoundationStereo while closely matching its zero-shot accuracy.
  • From the README, on an RTX 3090 at 640x480: 49.4 ms in PyTorch and 23.4 ms with TensorRT for the largest released checkpoint, down to 29.3 ms and 14.0 ms for the smallest, with peak memory around 650 MB. The model card lists 14.6 million parameters.

Limits

  • I read the abstract, README and model card only.
  • Needs rectified stereo and a known baseline to turn disparity into depth. A flat-port housing bends rays and breaks the rectified pinhole assumption unless it is modelled.
  • Nothing in what I read is underwater; the pseudo-labels come from internet stereo in air.

Relevance

Real-time stereo in about 650 MB is within reach of a Jetson-class vehicle, and stereo gives metric depth without a learned scale guess. Whether it holds up in backscatter and low texture is untested; compare cai-2025-underwater-metric-depth for the monocular case and she-2024-refractive-colmap for refraction. See foundation models for robotics.