Mobile robots first, then teams of them. Most of it was hands-on, on ROS 2, with small cars and a simulator.
Main ideas
- Feedback control, from PID loops up to tracking a planned path.
- Planning with graph search over occupancy grids.
- Camera calibration, features and place recognition.
- Kalman and particle filters for localisation, and SLAM.
- Consensus, task assignment and multi-robot path planning, where each robot’s choices start to depend on the others’.
- A short introduction to reinforcement learning, which I followed up in RL for Embodied AI.
Project
In a team of four, a search and rescue run with two small cars in an indoor arena. One car drove itself and a person drove the other, and the two shared what they found over a common message bus. Each car carried a Jetson, a ZED 2 stereo camera and an Arduino that drove the motors. We split the work four ways: localisation, planning, control and vision.
My part was most of the autonomous car’s software:
- a mission state machine (search, head for the exit, done),
- anchoring the car’s odometry to a map we built from a 3D scan of the arena,
- turning stereo depth into a forward laser scan, so the planner could avoid obstacles,
- an occupancy grid with A* routing,
- a small object detector trained on the target images, running on the car through OpenCV with no deep learning framework installed,
- a simulator with the same interfaces as the car, and a dashboard and replay tools to audit every run afterwards.
On the day the autonomous car finished its search and drove out through the exit. Most of what I learned came from the runs that went wrong before that:
- A node with full unit test coverage sat on the car for three hours doing nothing, because the launch file never passed it its configuration. Every test supplied the parameter by hand. The test that catches this checks the wiring: that the launch file gives every node what it needs.
- Recovery made things worse. When the car lost its pose mid-run, it relocalised twelve times in 90 seconds, and each attempt moved its map frame by 10 to 15 m. Holding still, or going back to a known spot and relocalising once, would have been better.
- The map was right but incomplete. The scan had barely looked at one end of the arena, so the map stopped where the room carried on.
- A localiser should refuse to answer when its evidence is thin. A confident answer on little evidence is the one that does damage.
Relevance to my research
A recovery that does more harm than the fault is the problem in failure recovery, and checking that components are wired up as well as correct belongs with runtime monitoring. Two cars that share findings are a small case of collaborative SLAM and of multi-robot perception.