INVARLLM: LLM-assisted Physical Invariant Extraction for Cyber-Physical Systems Anomaly Detection

3 min read

INVARLLM asks an LLM to read a plant’s documentation and propose physical invariants, then keeps only those the data supports. The LLM first builds a glossary of sensors and actuators, then, with a chain-of-thought prompt, writes hypothesised invariants as fuzzy function forms (which variables relate, roughly how). A validation step inspired by PCMCI+ causal discovery, with K-means over the scores, discards hallucinated or spurious relations that the logs do not show. Each surviving invariant is fitted as a regression, and an alarm fires when its error over a sliding window spikes.

Findings

  • Datasets: SWaT (36 attacks) and WADI (15 attacks).
  • Case-level (does each attack raise an alarm): 29 of 36 SWaT attacks and 13 of 15 WADI attacks detected, zero false alarms, so 100% precision and 80.56% and 86.67% recall.
  • Sample-level, the picture is weaker. Raw SWaT precision is 42.54% and recall 21.96% (FPR 4.10%). After the authors’ correction for alarm tails that run on just after an attack, 98.77% and 76.36%. WADI corrected: 77.93% and 67.83%.
  • Most SWaT false negatives come from one attack longer than 10 hours (62.6% of all anomalous samples), where the window sits fully inside the attack and the error stops spiking; alarms still fire at its start and end.
  • Hypothesised invariants before validation vary a lot by LLM: 40 (SWaT) and 42 (WADI) from GPT-o1, 16 and 33 from Claude 3.7 Sonnet, 23 and 20 from GPT-4o, 18 and 17 from DeepSeek; 80 and 74 unique in total.

Limits

  • The headline “100% precision, no false alarms” is the case-level metric. The raw sample-level numbers are much lower, which the authors report.
  • Needs good documentation. Plants or robots without written process descriptions get little from the LLM step.
  • Validation can only confirm relations present in the logged normal data, so it inherits the coverage problem of data-mined invariants.
  • The threat model assumes the attacker cannot compromise every device, and leaves passive eavesdroppers out of scope. Attackers may know the physical and control models, yet the evaluation uses only the fixed SWaT and WADI attack sets, and no attack was crafted to stay inside the learned invariants.
  • The authors list concept drift (temperature, humidity) as a source of false positives over time.

Relevance

It extends the line from hand-derived invariants through mined ones. Whether LLM-drafted invariants transfer to a moving platform, where “normal” depends on the task, is open. See cyber-physical security and runtime monitoring.