Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Deep neural network (DNN) potentials are an emerging tool for simulation of dynamical atomistic systems, with the promise of quantum mechanical accuracy at speedups of 10000 ×. As with other DNN methods, hyperparameters used during training can make a substantial difference in model accuracy, and optimal settings vary with dataset. To enable rapid tuning of hyperparameters for DNN potential training, we developed a scalable multiobjective optimization evolutionary algorithm for supercomputers and tested it on the Summit system at the Oak Ridge Leadership Computing Facility (OLCF). The multiobjective approach is required due to the coupling of two learned values defining the potential: the energy and force. Using a large-scale implementation of the NSGA-II algorithm adapted for training DNN potentials, we discovered several optimal multiobjective combinations, including best choices of activation functions, learning rate scaling scheme, and pairing of the two radial cutoffs used in the three dimensional descriptor function.

Original languageEnglish
Title of host publication52nd International Conference on Parallel Processing, ICPP 2023 - Workshops Proceedings
PublisherAssociation for Computing Machinery
Pages172-179
Number of pages8
ISBN (Electronic)9798400708435
DOIs
StatePublished - Aug 7 2023
Event52nd International Conference on Parallel Processing, ICPP 2023 - Workshops Proceedings - Salt Lake City, United States
Duration: Aug 7 2023Aug 10 2023

Publication series

NameACM International Conference Proceeding Series

Conference

Conference52nd International Conference on Parallel Processing, ICPP 2023 - Workshops Proceedings
Country/TerritoryUnited States
CitySalt Lake City
Period08/7/2308/10/23

Funding

Thanks to Steven Young and Guojing Cong (ORNL), Travis Johnston (Striveworks), and Josh Romero (NVIDIA) for insightful feedback and assistance. This work was supported by the Office of Materials and Chemical Technologies within the Office of Nuclear Energy, U.S. Department of Energy, and used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725 This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

  • evolutionary computation
  • high performance computing
  • hyperparameter optimization
  • machine learning
  • molecular simulation
  • multiobjective optimization
  • neural network potentials
  • neural networks

Fingerprint

Dive into the research topics of 'Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II'. Together they form a unique fingerprint.

Cite this