Abstract
Finding optimal hyperparameters is necessary to identify the best performing deep learning models but the process is costly. In this paper, we applied model-based optimization, also known as Bayesian optimization, using the CANDLE framework implemented on a High-Performance Computing environment. As a use case we selected information extraction from cancer pathology reports using a multi-task convolutional neural network, and hierarchical convolutional attention network to be optimized. We utilized a synthesized text corpus of 8,000 training cases and 2,000 validation cases with four types of clinical task labels including primary cancer site, laterality, behavior, and histological grade. We demonstrated that hyperparameter optimization using the CANDLE framework is a feasible approach with respect to both scalability and clinical task performance.
Original language | English |
---|---|
Title of host publication | 2019 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2019 - Proceedings |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
ISBN (Electronic) | 9781728108483 |
DOIs | |
State | Published - May 2019 |
Externally published | Yes |
Event | 2019 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2019 - Chicago, United States Duration: May 19 2019 → May 22 2019 |
Publication series
Name | 2019 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2019 - Proceedings |
---|
Conference
Conference | 2019 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2019 |
---|---|
Country/Territory | United States |
City | Chicago |
Period | 05/19/19 → 05/22/19 |
Funding
This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of the manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). ACKNOWLEDGMENT This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.
Keywords
- Convolutional neural networks
- Deep learning
- Hierarchical convolutional attention networks
- High-performance computing
- Hyperparameter optimization
- Model based optimization