AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification

Romain Egele, Romit Maulik, Krishnan Raghavan, Bethany Lusch, Isabelle Guyon, Prasanna Balaprakash

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

18 Scopus citations

Abstract

Deep neural networks are powerful predictors for a variety of tasks. However, they do not capture uncertainty directly. Using neural network ensembles to quantify uncertainty is competitive with approaches based on Bayesian neural networks while benefiting from better computational scalability. However, building ensembles of neural networks is a challenging task because, in addition to choosing the right neural architecture or hyperparameters for each member of the ensemble, there is an added cost of training each model. To address this issue, we propose AutoDEUQ, an automated approach for generating an ensemble of deep neural networks. Our approach leverages joint neural architecture and hyperparameter search to generate ensembles. We use the law of total variance to decompose the predictive variance of deep ensembles into aleatoric (data) and epistemic (model) uncertainties. We show that AutoDEUQ outperforms probabilistic backpropagation, Monte Carlo dropout, deep ensemble, distribution-free ensembles, and hyper ensemble methods on a number of regression benchmarks.

Original languageEnglish
Title of host publication2022 26th International Conference on Pattern Recognition, ICPR 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1908-1914
Number of pages7
ISBN (Electronic)9781665490627
DOIs
StatePublished - 2022
Externally publishedYes
Event26th International Conference on Pattern Recognition, ICPR 2022 - Montreal, Canada
Duration: Aug 21 2022Aug 25 2022

Publication series

NameProceedings - International Conference on Pattern Recognition
Volume2022-August
ISSN (Print)1051-4651

Conference

Conference26th International Conference on Pattern Recognition, ICPR 2022
Country/TerritoryCanada
CityMontreal
Period08/21/2208/25/22

Funding

V. ACKNOWLEDGEMENT This work was supported by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research, under Contract DE-AC02-06CH11357 and DOE Early Career Research Program award. We are grateful for the use of the computing resources in the Joint Laboratory for System Evaluation and Leadership Computing Facility at Argonne.

Fingerprint

Dive into the research topics of 'AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification'. Together they form a unique fingerprint.

Cite this