Catalogue

36 min read

Every page in the diary, one line each: 224 papers, 59 other notes and 34 entities. Generated by bun run diary index on 2026-10-16. For a table you can sort, see the reading list.

Start here

  • Research diary: Working notes from my PhD on resilient multi-robot systems, organised by robotics subfield.
  • Reading list: Every paper in the diary, as a table you can sort and filter by thread, domain and status.
  • Log: What was added to the diary and when, newest last. Append-only.
  • About this diary: How the diary is organised and kept up to date.
  • How this diary works: The structure of the diary, the kinds of pages in it, and how it is kept up to date.
  • Entities: Datasets, platforms, sensors, software, models and research groups that come up across the diary. Generated.
  • Thesis map: the three thesis threads on one page you can pan and zoom.

Thesis threads

  • Thesis threads: Three directions my thesis might take, each cutting across several subfields.
  • Long-term autonomy: A possible thesis thread: robots that keep working for weeks or months without a person resetting them.
  • Multi-robot multimodal perception: A possible thesis thread: robots that see the world through different sensors, on land, at sea and in the air, building one picture of it that a person can query and rely on.
  • Robust robotics: A possible thesis thread: robots and robot teams that keep working when some of their parts fail, are wrong, or are compromised.

Perception

Seeing the world from robot sensors: segmentation, depth, and the foundation models that now do much of both. Hub: Perception.

Papers (12):

  • Depth Anything 3 (2025, to-read): Depth Anything 3 predicts spatially consistent geometry from any number of images, with or without known poses.
  • Fast-FoundationStereo (2026, skimmed): Fast-FoundationStereo shrinks the FoundationStereo model until it runs in real time while keeping most of its zero-shot accuracy.
  • InfiniDepth (2026, skimmed): InfiniDepth represents depth as a neural implicit field: a local implicit decoder can be queried at any continuous 2D coordinate, so the output resolution is not tied to the input…
  • MoGe-2 (2025, skimmed): MoGe-2 extends MoGe from scale-free point maps to metric ones without losing the relative accuracy of the affine-invariant representation.
  • OS-Atlas (2024, skimmed): OS-Atlas is a foundation action model for agents that operate graphical interfaces, which have to ground an instruction to an on-screen element before acting on it.
  • Active perception (2018, skimmed): A historical review arguing that a complete artificial agent must include active perception: choosing how to move and where to look in order to perceive better.
  • Whisper (2023, skimmed): Whisper trains one speech recognition model on a very large set of weakly labelled audio, multilingual and multitask, and studies how well it works without fine-tuning.
  • SAM 2 (2024, skimmed): SAM 2 extends promptable segmentation from images to video.
  • SAM 3 (2025, skimmed): SAM 3 detects, segments and tracks every instance of a concept in images and video, where the concept is a short noun phrase, an image exemplar, or both.
  • SAM (2023, skimmed): SAM defines promptable segmentation: given a point, a box or another prompt, return a valid mask.
  • UniDepth (2024, skimmed): UniDepth predicts metric 3D points from a single image with no camera intrinsics given.
  • UniDepthV2 (2025, skimmed): UniDepthV2 keeps the design of UniDepth (self-promptable camera module, pseudo-spherical output, geometric invariance loss) and changes three things: an edge-guided loss for…

Mapping and localisation

SLAM, reconstruction and robust estimation: building a geometric map and knowing where you are in it. Hub: Mapping and localisation.

  • Robust estimation: Estimating a pose, a shape or a map when some of the measurements are wrong, and knowing when the answer can be trusted.

Papers (16):

  • BatSLAM 2.0 (2026, to-read): BatSLAM 2.0 is a sonar-only SLAM system for a robot carrying a biomimetic binaural sonar, modelled on echolocating bats.
  • CRISP (2025, to-read): CRISP estimates both the pose and the shape of an object from one RGB-D crop, without being told the object’s category.
  • Fast-SAM3D (2026, skimmed): Fast-SAM3D speeds up SAM 3D without retraining.
  • GNC (2020, to-read): GNC turns any least-squares solver into an outlier-robust one, without an initial guess.
  • LoGeR (2026, to-read): LoGeR scales feed-forward dense reconstruction to very long videos without any optimisation afterwards.
  • MapAnything (2026, to-read): MapAnything is one feed-forward transformer that takes one or more images plus any optional geometry (intrinsics, poses, depth, partial reconstructions) and regresses metric 3D…
  • MASt3R-SLAM (2025, to-read): MASt3R-SLAM is a real-time monocular dense SLAM system built around MASt3R, a network that predicts aligned pointmaps and matches for a pair of images.
  • MoGe-3 (2026, skimmed): MoGe-3 targets the over-smoothed thin structures and small objects that monocular geometry models still get wrong.
  • MoGe (2025, skimmed): MoGe predicts a 3D point map from a single image, up to an unknown global scale and shift (an affine-invariant point map).
  • Observability-aware trajectory optimisation (2016, read): This paper plans quadrotor trajectories so that the state estimator can actually pin down its own calibration states: accelerometer and gyroscope biases and the position of the…
  • Pow3R-SLAM (2026, read): Pow3R-SLAM turns MASt3R-SLAM into an RGB-D system by feeding sensor depth into the two-view network Pow3R as a conditioning input, instead of fusing depth into the map as geometry.
  • QuerySplat (2026, read): QuerySplat predicts a 3D Gaussian Splatting scene from a handful of unposed images in a single forward pass.
  • SAM 3D (2025, to-read): SAM 3D is a generative model that predicts an object’s geometry, texture and pose in the scene (layout) from a single image, aimed at cluttered natural photos where occlusion is…
  • TEASER++ (2021, to-read): TEASER registers two 3D point sets from putative correspondences when almost all of them are wrong.
  • VGGT (2025, to-read): VGGT is a feed-forward transformer that takes one, a few or hundreds of images and directly predicts camera parameters, point maps, depth maps and 3D point tracks.
  • VSLAM-LAB (2025, skimmed): VSLAM-LAB is a benchmarking framework for visual SLAM: one command-line tool that installs a baseline, downloads and preprocesses a dataset, runs a YAML-defined experiment for a…

Semantic mapping

Maps that know what things are: 3D scene graphs, open-vocabulary maps, and maps that change over time. Hub: Semantic mapping.

Papers (42):

  • 3-D Scene Graph (Kim) (2019, read): Kim et al.
  • 3D Dynamic Scene Graphs (2020, read): Rosinol et al.
  • 3D Scene Graph (Armeni) (2019, read): Armeni et al.
  • 3DSG survey (2026, read): This survey of 3D scene graphs is written by people from most of the groups that built the field (Stuttgart, Sapienza, Google, MIT, Freiburg, UTN, Montreal, TU Munich).
  • 3D VSG (2023, read): Looper et al.
  • CAPruner (2026, skimmed): CAPruner cuts down a 3D scene graph before an LLM reasons over it for 3D vision-language tasks such as referring-expression grounding and question answering.
  • ChronoGraph (2026, to-read): ChronoGraph is a functional 4D scene graph that links actions on object parts to the semantic and geometric changes they cause, using one form for both observed and predicted…
  • Clio (2024, read): Clio decides how finely to carve a scene into objects by looking at the task list.
  • ConceptGraphs (2023, read): ConceptGraphs builds an object-level 3D scene graph from posed RGB-D frames using only off-the-shelf 2D models, with no 3D training.
  • DAAAM (2026, skimmed): DAAAM builds a spatio-temporal memory of a large scene in real time, where each object carries a rich free-text description instead of a class label or a CLIP vector.
  • FOCUS (2026, to-read): FOCUS detects object-level change in semi-static scenes where an object may be swapped in place for a similar-looking one.
  • Hydra (IJRR) (2024, read): The journal version of Hydra adds the theory for why maps should be hierarchical and rebuilds two of Hydra’s heuristics.
  • FunGraph (2025, read): FunGraph adds a layer below objects to a ConceptGraphs-style scene graph: the knobs, handles, buttons and switches a robot actually touches, linked to their parent object by a…
  • 3DSG semantic uncertainty (2026, to-read): This framework gives each node of an open-vocabulary 3D scene graph a probability of being correct, using only the detector confidence and embeddings the graph already stores,…
  • HOV-SG (2024, read): HOV-SG builds a floor, room and object hierarchy with CLIP features at every level, offline, from RGB-D frames and odometry, and uses it to answer queries like “towel in the…
  • Hydra (2022, read): Hydra builds a five-layer 3D scene graph (mesh, objects and agents, places, rooms, building) online from visual-inertial data, where its predecessors Kimera and 3D DSGs needed…
  • Hydra++ (2026, read): Hydra++ replaces the centroid-and-bounding-box objects of Hydra and Khronos with full object meshes from a learned shape estimator.
  • Language-enabled spatial ontologies (2023, read): This paper builds the upper layer of a Hydra scene graph outdoors, where nobody has written down what the regions are and there is almost no labelled data.
  • Common-sense spatial knowledge (workshop) (2023, read): This four-page workshop paper is the first version of the logic tensor network idea later extended in strader-2023-ontology.
  • Kairos (2026, to-read): Kairos extends a hierarchical 3D scene graph into a 4D one that forecasts where people will be, and which way they will move, at times the robot has not observed.
  • Khronos (2024, read): Khronos builds a dense metric-semantic map of a scene at every past time, online, so a robot can answer “what was here at 1:52?” as well as “what is here now?”.
  • Kimera (2021, read): Kimera is the journal version that joins the Kimera metric-semantic SLAM library with 3D Dynamic Scene Graphs.
  • 3DSSG (2020, read): Wald et al.
  • H²-EMV (2026, to-read): H²-EMV lets a humanoid learn what to remember.
  • LOST-3DSG (2026, skimmed): LOST-3DSG keeps an open-vocabulary 3D scene graph up to date while objects move, without storing dense CLIP features.
  • Mono-Hydra (2023, skimmed): Mono-Hydra builds a Hydra-style 3D scene graph from a single camera plus an IMU, with no depth sensor.
  • Mono-Hydra++ (2026, to-read): Mono-Hydra++ is the follow-up to Mono-Hydra: still one RGB camera plus an IMU, but now a single multi-task network (M2H-MX, a frozen DINOv3 backbone with LoRA) predicts both depth…
  • Neural Trees (2021, to-read): A Neural Tree runs message passing on a tree built from the input graph.
  • OASIS-Map (2026, to-read): OASIS-Map keeps an object-level map consistent across repeated visits to a semi-static place, where objects appear, vanish, move or get replaced while the robot is away.
  • Outdoor open-set 3DSGs (2026, to-read): This field report tests open-set 3D scene graphs outdoors, using Terra 3DSG on five outdoor robot datasets.
  • Open3DSG (2024, read): Open3DSG predicts a scene graph straight from a 3D point cloud with class-agnostic instance masks, by distilling 2D vision-language features into a 3D graph neural network so that…
  • OpenGraph (2024, read): OpenGraph builds an open-vocabulary hierarchical graph of a driving-scale outdoor scene from RGB images, LiDAR scans and poses.
  • Picasso (2026, read): Picasso takes per-object pose and shape estimates (from SAM3D or CRISP) and corrects them so the whole scene is physically plausible: no two objects interpenetrate, nothing sinks…
  • UQ-DAAAM (2026, to-read): This paper adds object-level semantic uncertainty to VLM-based spatio-temporal memory, scored as how much an object’s captions disagree across views.
  • SceneGraphVLM (2026, skimmed): SceneGraphVLM has a small vision-language model write an image or video scene graph directly as text, in place of the usual detector-then-relation pipeline.
  • SEAM (2026, to-read): SEAM is a LiDAR lifelong mapping framework that anchors its evidence to submaps instead of one anchor for the whole session.
  • BiMoSG (2026, read): BiMoSG builds a 3D scene graph cheaply by default and pays for foundation models only where the task needs them.
  • S-Graphs (2022, read): S-Graphs puts the scene graph inside the SLAM factor graph, so walls, rooms and corridors are optimised together with the robot poses.
  • SuperMap (2026, read): SuperMap keeps an open-vocabulary, instance-level map of objects that stays valid as things are moved, removed or added, and exposes it as a “4D” scene graph a vision-language…
  • TrackGraph (2026, read): TrackGraph builds an open-vocabulary 3D scene graph online by keeping mask identities in the image stream before anything is fused into 3D.
  • vS-Graphs (2026, read): vS-Graphs is the camera version of S-Graphs: it adds two threads to ORB-SLAM3 so that walls, ground surfaces, rooms and floors become variables in the same local bundle adjustment…
  • Worth Remembering (2026, to-read): Worth Remembering uses Bayesian surprise to decide which episodes a robot keeps in long-term memory, computed in the latent space of V-JEPA 2 so it does not depend on the…

Planning and navigation

Deciding what to do and where to go: task planning on scene graphs, language-grounded planning and navigation. Hub: Planning and navigation.

  • Language-grounded planning: Turning an instruction such as “bring me something to drink from the kitchen” into a plan a robot can execute, using a map of the actual place.

Papers (14):

  • LSG (2024, read): Layered Semantic Graphs (LSG) builds a hierarchical scene graph top-down from the concepts an inspection planner uses.
  • ASHiTA (2025, to-read): ASHiTA takes a high-level instruction such as “prepare for dinner”, which may not name a single object, and breaks it into subtasks that are each tied to objects in a 3D scene…
  • CoSMO (2026, to-read): CoSMO turns the VLM embeddings in a semantic map into conformal coverage sets, then inflates obstacles by class so each hazard gets the clearance it needs.
  • EvoNav-Bench (2026, to-read): EvoNav-Bench extends GOAT-Bench style lifelong navigation to environments that change between tasks, built on ProcTHOR, so earlier experience is useful but no longer fully…
  • DSG navigation policies (2021, read): Ravichandran et al.
  • Multi-robot language-grounded planning (2025, to-read): This system joins several MIT pieces into one outdoor multi-robot pipeline.
  • LTR² (2026, to-read): LTR² is a teach-and-repeat system that teaches with LiDAR in good conditions and repeats with 4D millimetre-wave radar, which keeps working in bad weather and smoke.
  • LLM-guided scene graph planning (2023, read): Dai et al.
  • PaLM-E (2023, skimmed): PaLM-E feeds continuous sensor inputs (images and state estimates) into a pretrained large language model as “multi-modal sentences” that interleave encoded observations with…
  • PSG-Nav (2026, skimmed): PSG-Nav does open-vocabulary object-goal navigation over a 3D probabilistic scene graph.
  • SayPlan (2023, read): SayPlan plans long-horizon mobile manipulation tasks with GPT-4 over a pre-built 3D scene graph serialised as JSON (floors, rooms, assets, objects, with pose nodes linking rooms).
  • Taskography (2022, read): Taskography is a benchmark of 20 symbolic task planning domains (3734 tasks in total) written in PDDL over the 3D scene graphs of Gibson buildings, which average 2 to 3 floors and…
  • Nav2 (2020, skimmed): Navigation2 is the ROS 2 successor to the ROS Navigation Stack, which the authors say had not kept up with modern needs.
  • xFLIE (2024, read): xFLIE is the journal-length version of LSG: the same four-layer Target, Level, Pose, Feature graph (renamed 3DLSG) built incrementally by the FLIE inspect-explore planner, now…

Control

Keeping robots safe while they move: control barrier functions and safety filters for single robots and teams. Hub: Control.

  • Safe control: Keeping a robot inside a safe set (no collisions, no stalls over water, no battery below the return threshold) while it does its job.

Papers (4):

  • CBF-QP (2017, read): Ames, Xu, Grizzle and Tabuada define safety as forward invariance of a set and certify it with barrier functions instead of computing reachable sets.
  • Equivariant graph CBF (2025, read): Bousias, Lindemann and Pappas learn a distributed control barrier function and a policy for quadrotor swarms with a graph network, as in GCBF+, but force both to respect the…
  • MDBC (2021, to-read): Qin and colleagues learn two neural networks per agent at once: a controller and a control barrier function (CBF) that certifies it.
  • Safety barrier certificates (2016, read): Wang, Ames and Egerstedt add collision avoidance to any existing multi-robot controller as a minimally invasive filter.

Robot learning

Robot learning and the models behind it: policies, vision-language-action models, benchmarks, and evaluating language models. Hub: Robot learning.

Papers (28):

  • Embodied AI survey (2022, read): The survey compares nine embodied AI simulators (DeepMind Lab, AI2-THOR, CHALET, VirtualHome, VRKitchen, Habitat-Sim, iGibson, SAPIEN and ThreeDWorld) on seven features:…
  • LLM-as-a-Judge survey (2024, read): The survey defines an LLM judge as an evaluation produced by an LLM from the item to be judged combined with a context (prompt, history), and then asks how to make that evaluation…
  • Agentic Robot (2025, to-read): Agentic Robot targets error build-up in long-horizon manipulation.
  • AlphaEvolve (2025, read): AlphaEvolve improves programs by evolution, with LLMs writing the mutations.
  • Transformer (2017, read): The Transformer drops recurrence and convolution from sequence-to-sequence models and keeps only attention: an encoder and a decoder built from stacked self-attention and…
  • AutoScientists (2026, to-read): AutoScientists runs a decentralised team of AI agents on long computational experiments.
  • CaRR (2026, read): CaRR replaces the binary “is the final answer right” reward for web search agents with a reward for the evidence behind the answer.
  • Diffusion Policy (2023, to-read): Diffusion Policy represents a visuomotor policy as a conditional denoising diffusion process over robot actions, so the policy can represent several valid ways of doing the same…
  • EmboCoach-Bench (2026, read): EmboCoach-Bench tests whether LLM agents can do the engineering around robot learning: write dense rewards, fix training code, and tune diffusion policies, ACT and VLAs.
  • Feel-WM (2026, to-read): Feel-WM is an off-road navigation world model that conditions on proprioception and predicts what the robot will feel (future proprioceptive state and a failure risk) as well as…
  • Gemma (2024, skimmed): Gemma is a family of lightweight open models built from the research behind Gemini.
  • LalaEval (2024, read): LalaEval is a procedure for human evaluation of LLMs inside one company and domain, worked through for intracity freight at the logistics firm Huolala.
  • ACT (2023, to-read): The paper introduces Action Chunking with Transformers (ACT), an imitation learning method that learns a generative model over short sequences of actions (chunks), trained from…
  • LeRobot (2026, to-read): LeRobot is Hugging Face’s open-source library covering the whole robot learning stack: low-level motor control middleware, dataset collection, storage and streaming,…
  • LIBERO (2023, to-read): LIBERO is a benchmark for lifelong learning in robot manipulation, where a robot learns tasks one after another and should carry knowledge forward.
  • Calibration (2017, skimmed): A classifier is calibrated when its predicted probability matches how often it is right.
  • OpenVLA (2024, to-read): OpenVLA is an open 7-billion-parameter vision-language-action model trained on 970,000 real-world robot demonstrations.
  • Query by committee (1992, skimmed): Query by committee is an early active learning rule.
  • RoboTwin 2.0 (2025, read): RoboTwin 2.0 generates bimanual manipulation data in simulation and benchmarks policies on it.
  • RT-2 (2023, skimmed): RT-2 turns a vision-language model into a robot policy by writing robot actions as text tokens and co-fine-tuning on robot trajectories together with internet-scale…
  • Selective classification (2017, skimmed): A selective classifier may refuse to answer.
  • Self-consistency (2023, skimmed): Self-consistency replaces greedy decoding in chain-of-thought prompting: sample several reasoning paths from the same model and keep the answer most of them reach.
  • Deep ensembles (2017, skimmed): Deep ensembles estimate predictive uncertainty by training several networks independently and combining their predictions.
  • SmolVLA (2025, to-read): SmolVLA is a small vision-language-action model meant to be trained on one GPU and run on consumer GPUs or even CPUs.
  • T2-Z (2026, to-read): T2-Z adds a learnable environment latent to a robotic world model to capture hidden factors such as friction, mass or task rules, without labels for what those factors are.
  • Compression Principle (2026, read): A short position essay.
  • V-JEPA 2 (2025, to-read): V-JEPA 2 is a self-supervised joint-embedding predictive model pre-trained on over 1 million hours of internet video plus images.
  • π0 (2024, to-read): π0 puts a flow matching action head on a pretrained vision-language model, so the policy inherits web-scale semantics and outputs continuous actions.

Multi-robot systems

Teams of robots working on a shared task, and what happens when sharing fails. Hub: Multi-robot systems.

Papers (23):

  • Blockchain vs W-MSR (2020, read): Strobel, Castelló Ferrer and Dorigo compare three ways for 20 simulated robots in ARGoS to agree on the fraction of white tiles in a 2 m by 2 m arena: the linear consensus…
  • Decentralized Blocklist Protocol (2023, read): The Decentralized Blocklist Protocol (DBP) lets cooperative robots accuse a peer whenever a local observation contradicts what the peer reported.
  • CURB-SG (2024, read): CURB-SG builds a dynamic 3D scene graph of a city from several cars with panoptic lidar.
  • CERPE (2026, to-read): CERPE estimates relative pose between robots that meet briefly, over limited bandwidth, often with little or no shared view.
  • Vicsek model convergence (2003, read): Jadbabaie, Lin and Morse prove why the Vicsek flocking model works.
  • Crazyswarm (2017, read): Crazyswarm flies 49 Crazyflie 2.0 nano-quadcopters indoors in dense formation from one base station.
  • Discovering antagonists (2026, read): Wenger, Eberhard and Ebel detect robots that misuse a Voronoi coverage task to take over a region of interest.
  • Energy-constrained coverage (2025, read): Pant and colleagues treat battery recharging as a disturbance to coverage control.
  • Conformal map fusion (2026, to-read): Each robot builds a Gaussian-process occupancy map with a conformal coverage guarantee, but the guarantee degrades because samples along a trajectory are correlated and each robot…
  • FRAC-MARL (2026, to-read): FRAC-MARL is a decentralised actor-critic multi-agent RL method that tolerates Byzantine edge attacks, where the adversary corrupts messages on communication links.
  • HEROIC (2026, to-read): HEROIC coordinates a decentralised air-ground team for open-vocabulary search, with agents talking only in natural language.
  • Hickory (2026, read): Hickory gives each object node in a 3D scene graph four geometric layers, where most pipelines keep only a partial point cloud or a bounding box.
  • Hydra-Multi (2023, read): Hydra-Multi builds one 3D scene graph from a team of robots, online.
  • DeNeR pursuit-evasion (2025, read): A swarm of pursuer drones learns to track one faster, learning evader with MADDPG (centralised training, decentralised execution).
  • Multi-robot assembly survey (2018, skimmed): Marvel, Bostelman and Falco (NIST) survey industrial assembly done by two or more robots without fixtures: arms, dexterous hands and mobile platforms such as automated guided…
  • Role-based MADDPG pursuit (2023, read): This paper splits a drone swarm into two roles and trains both with a role-based variant of MADDPG.
  • PCM (2018, to-read): PCM decides which inter-robot loop closures to trust when two robots merge their pose graphs.
  • Reputation-based consensus (2026, to-read): This algorithm reaches Byzantine-resilient vector consensus without being told how many neighbours might be faulty.
  • Resilient task allocation (2021, read): Mayya and colleagues reassign a team of aerial and ground robots to tasks when weather, terrain or an attack degrades specific capabilities.
  • Blockchain token economy (2023, read): Strobel, Pacheco and Dorigo run an Ethereum blockchain on up to 24 Pi-puck robots and use a smart contract as the swarm’s shared memory for a collective-sensing task: estimate the…
  • ROMAN (2024, to-read): ROMAN (Robust Object Map Alignment Anywhere) relocalises a robot against its own old map or another robot’s map by aligning small maps of open-set objects.
  • SGM-SLAM (2026, to-read): SGM-SLAM is a distributed SLAM framework for robot teams with LiDAR, cameras and IMUs that finds inter-robot constraints by matching scene graphs.
  • PERSEPHONE perception (2026, to-read): This paper describes the perception pipeline linking two robots in the PERSEPHONE autonomous mining mission.

Safety and verification

Runtime monitoring, formal methods, failure detection and recovery: keeping robots correct while they run. Hub: Safety and verification.

  • Failure detectors compared: How recent detectors decide that a robot policy is failing, what they need to train, and how they set their alarms.
  • Failure recovery: What a robot does after something goes wrong.
  • Runtime monitoring: Checking, while the robot runs, that what it is doing satisfies a property: “never closer than a metre to a person”, “every request is answered within two seconds”.

Papers (29):

  • Brief account of RV (2009, skimmed): A short survey that defines runtime verification as checking whether a single observed execution satisfies a correctness property, and places it next to model checking and testing.
  • Timed automata (1994, read): Alur and Dill add real-valued clocks to finite automata.
  • FAIL-Detect (2025, to-read): FAIL-Detect detects failures of imitation-learning policies without any failure data, treating the task as sequential out-of-distribution detection.
  • C-3PO (2023, to-read): C-3PO estimates an object’s pose from a partial depth point cloud and also says whether to believe the answer.
  • AAPD model (2025, read): O’Keeffe targets gradual wear (motors losing efficiency, sensors losing range) instead of the sudden failures most swarm fault work injects into a few robots.
  • FailBench (2026, to-read): FailBench collects 2,197 manipulation attempts from 14 public sources (12 real, 2 simulated) and asks 13 VLM-based detectors whether each attempt succeeded.
  • CP-Cert (2026, to-read): Fast certifiable estimators solve locally and then certify the candidate with cheap linear algebra, but many robotics relaxations are degenerate and break that check.
  • Formal methods for robots survey (2019, read): A systematic survey of formal specification and verification for autonomous robots, covering 2007 to 2018.
  • PRISM swarm verification (2010, read): Konur, Dixon and Fisher model a foraging swarm in the PRISM probabilistic model checker and check properties of the whole swarm.
  • Hide-and-Seek (2026, to-read): Hide-and-Seek trains a failure detector for vision-language-action (VLA) policies from one success or failure label per trajectory, with no per-step annotation.
  • MCMAS (2017, read): MCMAS is a symbolic model checker for multi-agent systems.
  • MemoGuard (2026, to-read): MemoGuard checks episodic memories before a communication-limited robot reuses them, because a retrieved action can match the current context and still be unsafe (changed…
  • No Free Checker (2026, to-read): No Free Checker surveys about 150 verifiers for robot policies: success detectors, reward models, runtime monitors, safety filters and temporal-logic specifications.
  • Intermittent faults in swarms (2025, read): Oğuz and colleagues go after intermittent faults: errors that appear, vanish and come back, such as GPS that degrades sporadically in clutter.
  • RIWM (2026, to-read): This six-page perspective argues that high likelihood and visual fidelity do not ensure a world model keeps the evidence needed for safe decisions.
  • RoboChart (2019, read): RoboChart is a state-machine language for robot controllers: a small profile of UML state machines with an explicit robotic platform (its variables, events and operations),…
  • STL robustness (2010, read): Donzé and Maler give Signal Temporal Logic (STL) a real-valued semantics, so a monitor reports how far a trajectory is from satisfying or violating a formula as well as a yes or…
  • ROSMonitoring (2020, skimmed): ROSMonitoring adds runtime verification to ROS without tying it to one specification language.
  • ROS RV and field testing guidelines (2024, read): Caldas and colleagues write 20 guidelines for building ROS systems that can be runtime-verified and tested in the field: 8 for developers (provide APIs for querying system modes,…
  • LTL3 (2011, read): This paper sets out what a runtime monitor for a temporal logic formula should say about a finite, still growing trace.
  • Shielded RL (2018, read): A shield is a small reactive system, synthesised before training from a safety specification in temporal logic and a finite, conservative abstraction of the environment.
  • SAFE (2025, to-read): SAFE defines multitask failure detection: one detector that must work on tasks and environments it never saw in training.
  • Shield synthesis (2015, read): Bloem and colleagues synthesise a “shield” that sits on the outputs of a reactive hardware design and corrects them, without delay, so that a small set of critical safety…
  • SOTER (2019, read): SOTER is a publish-subscribe programming language for robot software with Simplex-style runtime assurance built in.
  • Temporal logic of programs (1977, read): Pnueli proposes reasoning about programs that are never meant to halt (operating systems, controllers) with a logic of time.
  • Hybrid automata (1996, read): An invited survey of the theory of hybrid automata, the standard model for a digital controller driving an analogue plant.
  • Verified AI (2022, read): A position paper on what formal verification of AI-based systems would take.
  • Simplex (2001, skimmed): Sha argues that building diverse, redundant versions of complex software does not by itself buy reliability, because a fixed budget split three ways gives three weaker versions.
  • Z3 (2008, skimmed): The four-page tool paper that introduced Z3, Microsoft Research’s satisfiability modulo theories (SMT) solver.

Security

Attacks on robots and the defences that hold up: spoofing, prompt injection, side channels, and invariants. Hub: Security.

  • Cyber-physical security: Security for systems where software controls physical processes: water treatment, power grids, robots.
  • Robot security: Attacks on robots, and defences that work when the attacker can touch the physical world.
  • SWaT testbed: Secure Water Treatment: a scaled-down, working water treatment plant at iTrust, SUTD, used for research and training in industrial control security.

Papers (22):

  • Attack-resilient estimation (2014, read): Bezzo and colleagues keep a ground robot’s cruise control working when one of its speed sensors is attacked.
  • BadVLA (2025, to-read): BadVLA plants backdoors in vision-language-action models.
  • CILC (2026, to-read): CILC addresses a threat from inside the team: a compromised robot can reconstruct approximations of an honest robot’s images and trajectory from the global descriptors it…
  • Confidence-weighted trust consensus (2025, read): Ballotta and colleagues make consensus resilient by using “trust observations” from the wireless channel, a value in [0, 1] per message that says how likely the sender is…
  • Detect and Suppress (2026, to-read): Detect and Suppress runs a sparse autoencoder on a VLA’s internal representations and finds a feature whose activation tracks the presence of an adversarial patch.
  • Distributed attack detection (SSMP) (2016, skimmed): Adepu and Mathur detect attacks that compromise two or more sensors or actuators in the same stage of a water treatment plant at once (single-stage multipoint, SSMP).
  • SAD DNS (2020, read): SAD DNS brings back off-path DNS cache poisoning by splitting one 32-bit guess (16-bit source port plus 16-bit transaction ID) into two 16-bit guesses.
  • SAD DNS resurrections (2021, read): After SAD DNS was patched, the same group used ICMP error messages, which by design never get a reply, as the probe.
  • SABER (2026, to-read): SABER is a two-tier Bayesian trust estimator for collaborative perception between connected vehicles.
  • Design vs data invariants (2020, skimmed): Umer and colleagues compare two ways to get the invariants that an attack detector checks in a water treatment plant.
  • Spoof-resilient networks (2017, read): Gil, Kumar and colleagues defend a multi-robot coverage team against a Sybil attack, where one malicious client forges many identities with fake positions to pull the servers away…
  • Physical prompt injection (2026, to-read): This study puts adversarial text in a robot’s camera view to hijack a VLM planner doing a sorting task.
  • INVARLLM (2024, read): INVARLLM asks an LLM to read a plant’s documentation and propose physical invariants, then keeps only those the data supports.
  • Range-based integrity monitoring (2025, read): Vijay and colleagues use inter-robot range measurements (UWB-style) as a cross-check on each robot’s own position estimate.
  • BWGNN (2022, read): Tang and colleagues look at graph anomalies through the graph spectrum.
  • RigidShift (2026, to-read): Swarm defences often cross-check GNSS positions against measured inter-drone distances.
  • SCAD (2025, read): SCAD treats a network side channel as a violation of non-interference: what an off-path attacker observes (a reply or not, or how a reply differs) should not depend on a victim’s…
  • Embodied agent security survey (2026, to-read): This survey sorts attacks on foundation-model-powered embodied agents by the first trust boundary the adversary crosses: five layers and twelve attack surfaces, from the model…
  • Semantic DoS (2026, read): Steinberg and Gal attack a robot that routes speech-to-text transcripts into a vision-language model which picks the next action.
  • Humanoid security SoK (2025, read): This systematisation of knowledge sorts published attacks and defences for humanoid robots into a seven-layer model: physical, sensing and perception, data processing, middleware,…
  • Trust-aware rollout (2026, to-read): This paper studies online multi-robot routing when some robots spoof their own localisation.
  • Multi-agent prompt injection (2026, to-read): This paper attacks an LLM-based multi-agent robot system with prompt injection, both directly through task instructions and indirectly through perception modules.

Human-robot interaction

People and robots: trust, interfaces, and how much an operator should rely on a robot team. Hub: Human-robot interaction.

Papers (1):

  • Brain2Qwerty v2 (2026, read): Brain2Qwerty v2 decodes typed sentences from MEG recordings alone.

Field robotics

Robots outside the lab, on land, at sea and in the air: underwater perception, aerial swarms, and Singapore’s settings. Hub: Field robotics.

  • Aerial swarms: Teams of small drones.
  • Singapore scenarios: Singapore has a busy port, strict airspace rules and a dense city on a small island, which gives useful test cases for robot teams:
  • Underwater perception: Seeing and localising under water, where most of what works on land stops working.

Papers (33):

  • Above and Below (2026, to-read): Above and Below is a centralised multi-robot SLAM system for one uncrewed surface vessel (USV) and several AUVs, built on inter-robot loop closures against structures visible both…
  • Sonar VFM adaptation (2026, to-read): This paper adapts vision foundation models trained on RGB to imaging sonar, since no large free sonar datasets exist to train an acoustic foundation model from scratch.
  • AGT-CV (2026, to-read): AGT-CV is a real air-ground dataset from a Clearpath Husky (3D LiDAR, stereo, IMU, GPS) and an Autel EVO II drone (RGB, thermal, GPS) on forest trails, rocky paths, mud, snow and…
  • Sea-thru distance accuracy (2024, read): Kuan and Jaw ask how accurate the camera-to-scene distance has to be before Sea-thru colour restoration works.
  • AQUA-SLAM (2025, to-read): AQUA-SLAM fuses a Doppler velocity log, a stereo camera and an IMU in one tightly coupled graph optimisation, for underwater scenes where visibility, light and visual features…
  • AquaFlow (2026, to-read): AquaFlow is a monocular, streaming Gaussian splatting SLAM system for underwater video.
  • AUSLUN (2026, to-read): AUSLUN uses a drone hovering near the coast, localising itself by visual-inertial odometry, as a long-range sensor and navigation anchor for an uncrewed surface vessel when GNSS…
  • Crazyflie 2.0 platform (2017, skimmed): This paper presents the Crazyflie 2.0 nano-quadrotor as a cheap, open-source and expandable platform for teaching and research.
  • DRACo-SLAM2 (2025, read): DRACo-SLAM2 finds inter-robot loop closures for a team of sonar-equipped underwater robots by matching small object maps first and pulling raw scans only after a match.
  • Eiffel Tower dataset (2023, to-read): The Eiffel Tower dataset benchmarks long-term visual localisation in the deep sea.
  • Underwater multi-modal spatial perception (2025, read): Kaveti et al.
  • GS underwater study (2026, to-read): This study runs five Gaussian splatting systems with public code under one protocol (shared poses, initialisation, compute budget and evaluator) across public underwater datasets…
  • Enhancement vs matching (2025, to-read): This paper judges underwater image enhancement by what it does for frame matching.
  • Benthic long-term localisation (2026, to-read): This paper releases a curated dataset for long-term visual localisation on the seafloor and a better way to define ground truth for it.
  • MAG-VLAQ (2026, to-read): MAG-VLAQ matches ground observations (camera and LiDAR) against aerial or satellite images for place recognition.
  • Mapping Pamir (2026, to-read): This paper maps the Pamir shipwreck off Barbados over several dives with an affordable action camera, adding water depth from a dive computer to the visual-inertial data.
  • MERLION (2025, read): MERLION picks a handful of frames from a long, murky AUV video that match what a user asked for in words.
  • Opti-acoustic semantic SLAM (2024, read): This paper builds object-level semantic SLAM for an underwater vehicle that is never told which object classes are in the scene.
  • Refractive COLMAP (2024, to-read): Refractive COLMAP builds refraction into every stage of COLMAP’s structure-from-motion pipeline, for both flat-port and dome-port underwater housings.
  • RUSplatting (2025, read): RUSplatting adapts 3D Gaussian splatting to sparse, noisy deep-sea imagery.
  • Sea-thru (2019, to-read): Sea-thru removes the water from an underwater photo using a range map of the scene.
  • SeaSplat (2025, to-read): SeaSplat constrains 3D Gaussian splatting with a physically grounded underwater image formation model, so range- and colour-dependent effects of the water are explained by the…
  • SeaThru-NeRF (2023, to-read): SeaThru-NeRF adds the water to the NeRF rendering equation.
  • Caustics removal via SLAM (2024, to-read): This paper removes caustics (the moving bright patterns sunlight casts on shallow seafloor) and backscatter from underwater images, without labels.
  • Swimm3R (2026, to-read): Swimm3R attacks the step that fails first in murky water, structure from motion.
  • TURTLMap (2024, to-read): TURTLMap localises and densely maps low-texture underwater scenes in real time on a low-cost vehicle, a BlueROV2 with an NVIDIA Jetson Orin Nano.
  • DM_underwater (2023, skimmed): This paper enhances underwater images with a conditional denoising diffusion model: the degraded image and Gaussian noise go in, and the network denoises towards a clean image.
  • Underwater metric depth benchmark (2025, to-read): This paper benchmarks zero-shot and fine-tuned monocular metric depth models on real underwater datasets that have metric depth ground truth, including FLSea and SQUID, across…
  • USS-Nav (2026, read): USS-Nav builds a four-layer scene graph (grid, topology, region, object) on board a small quadrotor and uses it for zero-shot object search, such as “find a sofa in the living…
  • UW-GS (2025, to-read): UW-GS adapts Gaussian splatting to water with three additions: a colour appearance model for distance-dependent colour change, a physics-based density control step that keeps…
  • Underwater enhancement and 3D review (2025, read): Huang et al.
  • Wat3R (2026, skimmed): Wat3R adapts a feed-forward 3D reconstruction model from air to water without any labelled underwater data.
  • WaterSplatting (2025, to-read): WaterSplatting pairs 3D Gaussian splatting for the geometry with a separate volumetric field for the water, queried once per pixel.

Entities

  • 3RScan: RGB-D scans of 478 indoor spaces, each rescanned over time as objects moved, 1,482 scans in all (Wald and others, 2019).
  • BlueROV2: Blue Robotics’ low-cost remotely operated underwater vehicle.
  • Clearpath Husky: Clearpath’s larger four-wheeled outdoor robot.
  • Clearpath Jackal: A small four-wheeled outdoor robot from Clearpath, common in field tests of mapping and navigation.
  • CLIP: OpenAI’s contrastive image and text model (Radford and others, 2021).
  • COLMAP: Structure-from-motion and multi-view stereo software (Schönberger and Frahm, 2016).
  • DINO: Meta’s self-supervised vision transformers (DINO, DINOv2, DINOv3).
  • Doppler velocity log: An acoustic sensor that measures an underwater vehicle’s velocity over the seabed.
  • FastSAM: A fast segment-anything model built on a YOLO-style detector (Zhao and others, 2023), used where SAM is too slow for a robot.
  • Franka arms: Franka Robotics’ seven-joint arms (Panda, FR3), the usual arm in manipulation and policy learning papers.
  • Gaussian splatting: A scene stored as millions of small 3D Gaussians that can be rendered in real time (Kerbl and others, 2023).
  • Gazebo: The open-source robot simulator most ROS projects use.
  • Gemini: Google DeepMind’s multimodal models, used as planners and failure judges in several of these papers.
  • GPT-4: OpenAI’s large language and vision-language models.
  • Habitat: Meta’s simulator for embodied AI, built for navigation in scanned buildings such as HM3D.
  • Intel RealSense: Intel’s RGB-D cameras, such as the D435 and D455.
  • iTrust: SUTD’s centre for research in cyber security, home of the SWaT, WADI and EPIC testbeds, where I worked from 2018 to 2026.
  • Kimera-Multi: Distributed multi-robot metric-semantic SLAM from MIT SPARK (Tian and others, 2022), the multi-robot version of Kimera.
  • Lidar: Laser range sensing.
  • Linear temporal logic: A logic for properties of executions over time: “always”, “eventually”, “until”.
  • MARVL Lab: My lab at SUTD, led by Malika Meghjani.
  • Matterport3D and HM3D: Matterport3D has RGB-D panoramas of 90 building-scale scenes (Chang and others, 2017).
  • MIT SPARK Lab: Luca Carlone’s lab at MIT.
  • Neural radiance fields: A scene stored as a neural network that maps a 3D point and a viewing direction to colour and density (Mildenhall and others, 2020).
  • NVIDIA Jetson: NVIDIA’s embedded GPU computers (Xavier NX, Orin Nano, Orin NX, Orin AGX), the usual onboard computer on small robots.
  • ORB-SLAM3: Feature-based visual and visual-inertial SLAM (Campos and others, 2021), the usual baseline for camera SLAM.
  • PDDL: The Planning Domain Definition Language (McDermott and others, 1998), the standard input format for classical task planners.
  • RANSAC: Random sample consensus (Fischler and Bolles, 1981): fit a model to random minimal subsets of the data and keep the one most points agree with.
  • Replica: Photorealistic reconstructions of 18 indoor scenes from Meta Reality Labs (Straub and others, 2019).
  • ROS: The Robot Operating System: middleware for passing messages between a robot’s processes.
  • ScanNet: RGB-D video of indoor rooms with 3D reconstructions and semantic labels, about 1,500 scans (Dai and others, 2017).
  • Spot: Boston Dynamics’ quadruped.
  • uHumans2: Simulated indoor and outdoor scenes with moving people, from MIT SPARK, released with the 3D dynamic scene graph work.
  • Unitree robots: Unitree’s quadrupeds and humanoids (Go2, B2, G1, H1), now common in legged robotics because they are cheap.

Courses

  • Foundations of Robotics and Multi-Robot Systems: Mobile robots first, then teams of them.
  • Course notes: Notes from PhD coursework at SUTD.
  • Multimodal Generative AI: Generative models for text, images and audio, from the transformer up to systems that take several modalities at once.
  • Networked Embodied AI: Foundation models that let a robot see, reason and act from vision and language, and the network that has to carry all of it on a budget of bandwidth, latency and compute.
  • Reinforcement Learning for Embodied AI: Reinforcement learning for agents that act in physical or simulated worlds, from tabular methods to deep ones, with robots as the running example.
  • Robotics Enterprise: How a robot gets from a prototype to something people use and pay for: who it’s for, what it replaces, and what it takes to run it outside a lab.
  • Secure Cyber-Physical Systems: Security for industrial control systems: the plants that move water, power and chemicals under the control of programmable logic controllers.
  • Systems Security: How attackers and defenders think about whole systems, from the memory of one process to the trust model of the web.

Dated entries

  • 2026-10-01 · Diary set up: Moved the research notes onto the website so they can link to the letters, papers and explainers.
  • 2026-10-16 · Reading sweep: Went through every paper on my disks, wrote a note for each, and put them in one reading list: 95 read properly, 40 skimmed from the abstract and code, and 89 to read.
  • Dated entries: Dated notes from the PhD.