Abstract
Deep-learner hyper-parameters, such as kernel sizes, batch sizes, and learning rates, can significantly influence the quality of trained models. The state of the art for finding optimal hyper-parameters generally uses a brute force, grid search approach, random search, or Bayesian-based optimization among other techniques. We applied an evolutionary algorithm to optimize kernel sizes for a convolutional neural network used to detect settlements in satellite imagery. Usually convolutional layer kernel sizes are small - typically one, three, or five - but we found that the system converged at, or near, kernel sizes of nine for the last convolutional layer, and that this occurred for multiple runs using two different datasets. Moreover, the larger kernel sizes had fewer false positives than the 3x3 kernel sizes found as optimal via a brute force uniform grid search. This suggests that this large kernel size may be leveraging patterns found in larger areal features in the source imagery, and that this may be generalized as possible guidance for similar remote sensing deep-learning tasks.
Original language | English |
---|---|
Title of host publication | Proceedings of DLS 2019 |
Subtitle of host publication | Deep Learning on Supercomputers - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 36-44 |
Number of pages | 9 |
ISBN (Electronic) | 9781728160115 |
DOIs | |
State | Published - Nov 2019 |
Event | 3rd IEEE/ACM Workshop on Deep Learning on Supercomputers, DLS 2019 - Denver, United States Duration: Nov 17 2019 → … |
Publication series
Name | Proceedings of DLS 2019: Deep Learning on Supercomputers - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis |
---|
Conference
Conference | 3rd IEEE/ACM Workshop on Deep Learning on Supercomputers, DLS 2019 |
---|---|
Country/Territory | United States |
City | Denver |
Period | 11/17/19 → … |
Funding
This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/ doe-public-access-plan).
Keywords
- Deep learning
- Evolutionary algorithms
- Hyper-parameters
- Optimization
- Remote sensing
- Settlement mapping