Selective Classification for Deep Neural Networks

1 min read

A selective classifier may refuse to answer. Here the user sets a target risk (error rate), and the method builds a rejection rule for a trained deep network so that, with high probability, the error on the inputs it does answer stays at or below that target. The cost is coverage: the share of inputs it answers.

Findings

  • On ImageNet, a top-5 error of 2% is guaranteed with 99.9% probability at almost 60% coverage.
  • CIFAR experiments are also reported; the abstract gives no figures for them.

Limits

  • Abstract-level reading only.
  • The abstract frames the guarantee as holding with high probability; it does not discuss what happens when test data drifts away from the data used to set the threshold.

Relevance

Abstaining when unsure follows this risk-coverage framing, and GoGet applies it by abstaining when its segmenter and reasoner disagree. For robust robotics the open question is the distribution shift case: for a robot in murky water or at dusk, a threshold set on clean data stops meaning what it promised. See runtime-monitoring.