Deep ensembles estimate predictive uncertainty by training several networks independently and combining their predictions. The authors offer it as an alternative to Bayesian neural networks that needs few changes to training, parallelises easily and needs little tuning.
Findings
- Uncertainty estimates are well calibrated, as good as or better than approximate Bayesian neural networks on classification and regression benchmarks.
- Under dataset shift the ensemble is more uncertain on out-of-distribution examples than on in-distribution test data.
- Scales to ImageNet.
Limits
- Abstract-level reading only.
- Costs several full training runs and several forward passes at test time.
Relevance
Spread among ensemble members is the uncertainty signal. GoGet uses disagreement between two different models, and the open question is which gives more independent errors: seeds of one architecture, or models of different families trained on different data. For a robot team the same question applies to teammates with different sensors, a thread in multi-robot multimodal perception and robust-estimation.
Links
- arXiv
- Subfield: Robot learning, Safety and verification