Multi-Agent Dynamically Networked and Decentralized Pursuit-Evasion

2 min read

A swarm of pursuer drones learns to track one faster, learning evader with MADDPG (centralised training, decentralised execution). The twist is that communication exists only during training: a Decentralized Networking Reward (DeNeR) pays each pursuer when the target is seen by a neighbour, a neighbour’s neighbour, or one hop further, with weights 1, 0.7 and 0.4 that the authors picked empirically. Neighbours come either from a coverage-range rule (everyone within a distance threshold) or from k-nearest neighbours. At deployment the pursuers stay in radio silence and act on local observations only, which the authors motivate by noisy links and the risk of jamming or spoofing.

Findings

  • Simulation with 10 pursuers, 1 evader that is 30% faster, and 3 obstacles. In a 2 m × 2 m arena the average distance to the target fell from 1.23 m without networking to 0.64 m with the coverage-range reward and 1.06 m with kNN (k = 2); the minimum distance fell from 0.38 m to 0.24 m and 0.20 m.
  • In a 10 m × 10 m arena the baseline learned nothing useful, while the minimum distance to target was 0.39 m (coverage range) and 0.36 m (kNN) against 0.89 m for the baseline.
  • kNN training converged around 20k episodes; the coverage-range version fluctuated until about 40k.
  • Real flights used five Crazyflie 2.1 pursuers and one evader with UWB positioning in a 6 m × 6 m area with three cones as obstacles. The real-world result is qualitative (screenshots and trajectories).
  • These numbers come from the version of the paper I read, which listed six authors; the published record lists seven.

Limits

  • 2D motion at constant speeds, one target, reward weights tuned by hand.
  • No quantitative metric for the real flights.
  • The learned joint policy assumes every teammate behaves as it did in training. A broken or adversarial pursuer was not studied.

Relevance

Radio silence at deployment removes the channel a liar would use, which is one answer to spoofing in aerial swarms. The price is that each robot’s model of its teammates is frozen at training time. What happens to such a policy when one pursuer stops behaving as trained is an open question for robust robotics and multi-robot systems; see kouzehgar-2023-multi-target-pursuit for the same group’s multi-target version.