Evolving larger convolutional layer kernel sizes for a settlement detection deep-learner on summit

Mark Coletti, Dalton Lunga, Jeffrey K. Bassett, Amy Rose

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Deep-learner hyper-parameters, such as kernel sizes, batch sizes, and learning rates, can significantly influence the quality of trained models. The state of the art for finding optimal hyper-parameters generally uses a brute force, grid search approach, random search, or Bayesian-based optimization among other techniques. We applied an evolutionary algorithm to optimize kernel sizes for a convolutional neural network used to detect settlements in satellite imagery. Usually convolutional layer kernel sizes are small - typically one, three, or five - but we found that the system converged at, or near, kernel sizes of nine for the last convolutional layer, and that this occurred for multiple runs using two different datasets. Moreover, the larger kernel sizes had fewer false positives than the 3x3 kernel sizes found as optimal via a brute force uniform grid search. This suggests that this large kernel size may be leveraging patterns found in larger areal features in the source imagery, and that this may be generalized as possible guidance for similar remote sensing deep-learning tasks.

Original languageEnglish
Title of host publicationProceedings of DLS 2019
Subtitle of host publicationDeep Learning on Supercomputers - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages36-44
Number of pages9
ISBN (Electronic)9781728160115
DOIs
StatePublished - Nov 2019
Event3rd IEEE/ACM Workshop on Deep Learning on Supercomputers, DLS 2019 - Denver, United States
Duration: Nov 17 2019 → …

Publication series

NameProceedings of DLS 2019: Deep Learning on Supercomputers - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference3rd IEEE/ACM Workshop on Deep Learning on Supercomputers, DLS 2019
Country/TerritoryUnited States
CityDenver
Period11/17/19 → …

Funding

This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/ doe-public-access-plan).

Keywords

  • Deep learning
  • Evolutionary algorithms
  • Hyper-parameters
  • Optimization
  • Remote sensing
  • Settlement mapping

Fingerprint

Dive into the research topics of 'Evolving larger convolutional layer kernel sizes for a settlement detection deep-learner on summit'. Together they form a unique fingerprint.

Cite this