How a robot gets from a prototype to something people use and pay for: who it’s for, what it replaces, and what it takes to run it outside a lab. I audited it.
Main ideas
- Start from the user and the place the robot will work, before fixing the design.
- A robot in the field needs operations and support: someone has to set it up, fix it and update it.
- The economics decide which research robots ever leave the demo stage.
Project
In a team of four, LocalEyes: a talking walking cane for blind and elderly walkers. A phone strapped to a cane describes the surroundings, reads signs, warns about obstacles and drop-offs with distances in metres, remembers places the user names, and guides them along routes it has been taught. There is no screen. Everything happens through voice and taps on the cane: two taps to talk, three to repeat, and safety alerts interrupt everything else.
A cane camera is a hard sensor. The cane sweeps from side to side, which wrecks ordinary frame-to-frame visual odometry. So LocalEyes takes a frame only at the end of each sweep, when the gyro is quiet and the image is sharp, counts steps from the cane’s strikes on the ground, smooths the compass over the sweep, and corrects its drift whenever it recognises a place it has seen before. A vision-language model (Gemma) does the describing, a depth model (Depth Anything 3) gives distances, and DINOv3 features recognise places. Indoors, it can navigate a building from a 3D scan.
Most of the changes came from field walks. Replaying the first runs showed the phone being handed 355 seconds of speech during a 121-second walk, which no listener can keep up with. Before the first walk with a blind tester we added a calmer setting and paced the speech: about a sixth fewer hazard announcements, with collision warnings unchanged. Indoors, the place recognition struggled to get a first fix, and reading room numbers off door signs became a second source of position.
Relevance to my research
An assistive device is trust in its plainest form: the user has to act on what it says, without being able to check. It is also a long-term autonomy problem, since it has to keep a map of the user’s places honest across weeks, and a failure recovery one, since it has to say when it is lost.