SAM defines promptable segmentation: given a point, a box or another prompt, return a valid mask. The authors train a promptable model alongside a data engine in which the model helps annotators, and release SA-1B, with over 1 billion masks on 11 million licensed, privacy-respecting images. Because it is promptable, the model transfers zero-shot to new image distributions and tasks.
Findings
- SA-1B: over 1 billion masks on 11 million images, the largest segmentation dataset at the time.
- The abstract reports zero-shot results that are often competitive with or better than earlier fully supervised methods. I have only read the abstract, so no task-level numbers here.
Limits
- Abstract-level reading only.
- Masks carry no class names. Open-vocabulary use needs another model to supply the prompt or the label, which SAM 3 later builds in.
- Nothing in the abstract about underwater, aerial or thermal imagery.
Relevance
The SAM line is the segmenter half of the segmenter and reasoner cross-check in GoGet. For open-vocabulary-mapping and multi-robot multimodal perception, the open question is how mask quality degrades at night or from altitude, where the training photos never went.
Links
- arXiv · project page
- Subfield: Perception