Whisper trains one speech recognition model on a very large set of weakly labelled audio, multilingual and multitask, and studies how well it works without fine-tuning.
Findings
- Trained on 680,000 hours of multilingual, multitask weak supervision.
- Works zero-shot on standard benchmarks without fine-tuning, and the authors say it approaches human accuracy and robustness.
- Models and inference code are released.
Limits
- Abstract-level reading only.
Relevance
Speech is the main input channel for an assistive robot, and language is one of the modalities in multi-robot multimodal perception. Open question: how does recognition hold up on a noisy deck, in wind or next to an engine, and can a robot tell when its transcript is wrong before acting on it? Cited in GoGet, which uses a different recogniser.
Links
- arXiv
- Subfield: Perception, Human-robot interaction