Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra

Daniel Martinez; Wesley Brewer; Gregory Behm; Andrew Strelzoff; Andrew Wilson; Daniel Wade

doi:10.1109/MLHPC.2018.8638645

Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra

Daniel Martinez, Wesley Brewer, Gregory Behm, Andrew Strelzoff, Andrew Wilson, Daniel Wade

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Scopus citations

Abstract

A method for Deep Neural Network (DNN) hyperparameter search using evolutionary optimization is proposed for nonlinear high-dimensional multivariate regression problems. Deep networks often lead to extensive hyperparameter searches which can become an ambiguous process due to network complexity. Therefore, we propose a user-friendly method that integrates Dakota optimization library, TensorFlow, and Galaxy HPC workflow management tool to deploy massively parallel function evaluations in a Genetic Algorithm (GA). Deep Learning Evolutionary Optimization (DLEO) is the current GA implementation being presented. Compared with random generated and hand-tuned models, DLEO proved to be significantly faster and better searching for optimal architecture hyperparameter configurations. Implementing DLEO allowed us to find models with higher validation accuracies at lower computational costs in less than 72 hours, as compared with weeks of random and manual search. Moreover, DLEO parallel coordinate plots provided valuable insights about network architecture designs and their regression capabilities.

Original language	English
Title of host publication	Proceedings of MLHPC 2018
Subtitle of host publication	Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	57-66
Number of pages	10
ISBN (Electronic)	9781728101804
DOIs	https://doi.org/10.1109/MLHPC.2018.8638645
State	Published - Jul 2 2018
Externally published	Yes
Event	2018 IEEE/ACM Machine Learning in HPC Environments, MLHPC 2018 - Dallas, United States Duration: Nov 12 2018 → …

Publication series

Name	Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	2018 IEEE/ACM Machine Learning in HPC Environments, MLHPC 2018
Country/Territory	United States
City	Dallas
Period	11/12/18 → …

Funding

This material is based upon work supported by, or in part by, the Department of Defense High Performance Computing Modernization Program (HPCMP) under User Productivity, Technology Transfer, and Training (PETTT) contract # GS04T09DBC0017. Any opinions, finding and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DoD HPCMP.

Funders	Funder number
Department of Defense High Performance Computing Modernization Program
HPCMP	GS04T09DBC0017

Keywords

Genetic algorithms
High performance computing (HPC)
Hyperparameter tuning
Neural architecture search (NAS)

Access to Document

10.1109/MLHPC.2018.8638645

Cite this

Martinez, D., Brewer, W., Behm, G., Strelzoff, A., Wilson, A., & Wade, D. (2018). Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra. In Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 57-66). Article 8638645 (Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MLHPC.2018.8638645

Martinez, Daniel ; Brewer, Wesley ; Behm, Gregory et al. / Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra. Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 57-66 (Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{1411d685f36b45cfa6d8354b7d42bfab,

title = "Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra",

abstract = "A method for Deep Neural Network (DNN) hyperparameter search using evolutionary optimization is proposed for nonlinear high-dimensional multivariate regression problems. Deep networks often lead to extensive hyperparameter searches which can become an ambiguous process due to network complexity. Therefore, we propose a user-friendly method that integrates Dakota optimization library, TensorFlow, and Galaxy HPC workflow management tool to deploy massively parallel function evaluations in a Genetic Algorithm (GA). Deep Learning Evolutionary Optimization (DLEO) is the current GA implementation being presented. Compared with random generated and hand-tuned models, DLEO proved to be significantly faster and better searching for optimal architecture hyperparameter configurations. Implementing DLEO allowed us to find models with higher validation accuracies at lower computational costs in less than 72 hours, as compared with weeks of random and manual search. Moreover, DLEO parallel coordinate plots provided valuable insights about network architecture designs and their regression capabilities.",

keywords = "Genetic algorithms, High performance computing (HPC), Hyperparameter tuning, Neural architecture search (NAS)",

author = "Daniel Martinez and Wesley Brewer and Gregory Behm and Andrew Strelzoff and Andrew Wilson and Daniel Wade",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 IEEE/ACM Machine Learning in HPC Environments, MLHPC 2018 ; Conference date: 12-11-2018",

year = "2018",

month = jul,

day = "2",

doi = "10.1109/MLHPC.2018.8638645",

language = "English",

series = "Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "57--66",

booktitle = "Proceedings of MLHPC 2018",

}

Martinez, D, Brewer, W, Behm, G, Strelzoff, A, Wilson, A & Wade, D 2018, Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra. in Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis., 8638645, Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 57-66, 2018 IEEE/ACM Machine Learning in HPC Environments, MLHPC 2018, Dallas, United States, 11/12/18. https://doi.org/10.1109/MLHPC.2018.8638645

Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra. / Martinez, Daniel; Brewer, Wesley; Behm, Gregory et al.
Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2018. p. 57-66 8638645 (Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra

AU - Martinez, Daniel

AU - Brewer, Wesley

AU - Behm, Gregory

AU - Strelzoff, Andrew

AU - Wilson, Andrew

AU - Wade, Daniel

PY - 2018/7/2

Y1 - 2018/7/2

N2 - A method for Deep Neural Network (DNN) hyperparameter search using evolutionary optimization is proposed for nonlinear high-dimensional multivariate regression problems. Deep networks often lead to extensive hyperparameter searches which can become an ambiguous process due to network complexity. Therefore, we propose a user-friendly method that integrates Dakota optimization library, TensorFlow, and Galaxy HPC workflow management tool to deploy massively parallel function evaluations in a Genetic Algorithm (GA). Deep Learning Evolutionary Optimization (DLEO) is the current GA implementation being presented. Compared with random generated and hand-tuned models, DLEO proved to be significantly faster and better searching for optimal architecture hyperparameter configurations. Implementing DLEO allowed us to find models with higher validation accuracies at lower computational costs in less than 72 hours, as compared with weeks of random and manual search. Moreover, DLEO parallel coordinate plots provided valuable insights about network architecture designs and their regression capabilities.

AB - A method for Deep Neural Network (DNN) hyperparameter search using evolutionary optimization is proposed for nonlinear high-dimensional multivariate regression problems. Deep networks often lead to extensive hyperparameter searches which can become an ambiguous process due to network complexity. Therefore, we propose a user-friendly method that integrates Dakota optimization library, TensorFlow, and Galaxy HPC workflow management tool to deploy massively parallel function evaluations in a Genetic Algorithm (GA). Deep Learning Evolutionary Optimization (DLEO) is the current GA implementation being presented. Compared with random generated and hand-tuned models, DLEO proved to be significantly faster and better searching for optimal architecture hyperparameter configurations. Implementing DLEO allowed us to find models with higher validation accuracies at lower computational costs in less than 72 hours, as compared with weeks of random and manual search. Moreover, DLEO parallel coordinate plots provided valuable insights about network architecture designs and their regression capabilities.

KW - Genetic algorithms

KW - High performance computing (HPC)

KW - Hyperparameter tuning

KW - Neural architecture search (NAS)

UR - http://www.scopus.com/inward/record.url?scp=85063061793&partnerID=8YFLogxK

U2 - 10.1109/MLHPC.2018.8638645

DO - 10.1109/MLHPC.2018.8638645

M3 - Conference contribution

AN - SCOPUS:85063061793

T3 - Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 57

EP - 66

BT - Proceedings of MLHPC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 IEEE/ACM Machine Learning in HPC Environments, MLHPC 2018

Y2 - 12 November 2018

ER -

Martinez D, Brewer W, Behm G, Strelzoff A, Wilson A, Wade D. Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra. In Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2018. p. 57-66. 8638645. (Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/MLHPC.2018.8638645

Deep Learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this