If at First You Don't Succeed, Backtrack: Lightweight Recovery for Frozen VLAs

2 min read

Backtrack adds failure detection and recovery to a VLA without retraining it, collecting recovery demonstrations, or running a second model. A small head reads the frozen policy’s internal representations and predicts whether the episode will fail; it is trained only on outcome-labelled rollouts of the same policy. When the predicted failure probability crosses a threshold, a controller pauses the policy, retraces the arm’s recent motion, and hands control back so the unchanged policy can try again from an earlier configuration.

Findings

  • The head adds 0.48 to 0.88% to the policy’s parameter count.
  • Tested on LIBERO with SmolVLA, π0.5 and π0-FAST, which covers flow-matching and autoregressive action heads.
  • On LIBERO-Long, where all three fail most, Backtrack cuts each policy’s failure rate by 19 to 48%, raising success by up to 7.4 points. On the three shorter suites, where success is already 85 to 99%, the effect is smaller.
  • On MetaWorld, all twelve controller configurations tested with SmolVLA improve on the unassisted policy.

Limits

  • Simulation only. No physical robot (authors).
  • Retracing assumes the motion can be reversed. It cannot help when the failure has made the task physically impossible, or when recovery needs a new strategy (authors).
  • The experiments do not separate why a retry succeeds: approximate return, a changed scene, or different sampled actions (authors).

Relevance

A clean data point for failure-recovery: much of the gain comes from detect, rewind and retry, with no learned recovery policy. Detection reads policy features as in SAFE and Progress Probes, so it fits comparison-failure-detectors. Rewinding is a recovery that a robot in a team could also announce to its teammates, which matters if they plan around where it said it would be.