Safety filters on humanoids promise collision-free control, but only when their safety parameters suit the task and the environment, and there is no general way to set them. Filters also optimise only the current step. They can fix the present violation while steering the robot into a state from which no safe control exists. The paper turns this into an attack that needs no special access: a user submits ordinary task requests, choosing intermediate goals so that the robot ends up where its filter cannot prevent a collision. SIREN automates the search for such “goal insertion” attacks with the cross-entropy method. The authors frame it as a hospital robot asked by a visitor to fetch something on the way.
Findings
- Attacks succeed against all seven safety filter algorithms evaluated, in simulation and on a Unitree G1.
- SIREN finds attacks in every vulnerable scenario of the SPARK safety benchmark. Over 35 vulnerable scenarios it finds 1.86 times as many attacks as random search (the abstract rounds this to three times as many as a baseline).
- On the real humanoid, the attack makes the robot collide while carrying out its task.
Limits
- The attacker knows the filter’s algorithm and parameter values. Attacks with partial knowledge are future work (authors).
- No estimate of how often a successful attack exists for an arbitrary task and environment (authors).
- Perception and SLAM are left out; errors there could help or hinder the attack (authors).
- A full defence needs predictive safe control with feasibility guarantees, still an open problem (authors).
Relevance
A direct challenge to the guarantees in safe-control: the CBF-QP certificate holds only for the parameters it was given, and the filter’s own short horizon becomes the attack surface. It fits robot-security as an attack through legitimate inputs, like semantic denial of service. For a team, an insider teammate could issue such goal requests too, which ties it to byzantine-fault-tolerance.