Abstract
Deep neural network (DNN) potentials are an emerging tool for simulation of dynamical atomistic systems, with the promise of quantum mechanical accuracy at speedups of 10000 ×. As with other DNN methods, hyperparameters used during training can make a substantial difference in model accuracy, and optimal settings vary with dataset. To enable rapid tuning of hyperparameters for DNN potential training, we developed a scalable multiobjective optimization evolutionary algorithm for supercomputers and tested it on the Summit system at the Oak Ridge Leadership Computing Facility (OLCF). The multiobjective approach is required due to the coupling of two learned values defining the potential: the energy and force. Using a large-scale implementation of the NSGA-II algorithm adapted for training DNN potentials, we discovered several optimal multiobjective combinations, including best choices of activation functions, learning rate scaling scheme, and pairing of the two radial cutoffs used in the three dimensional descriptor function.
Original language | English |
---|---|
Title of host publication | 52nd International Conference on Parallel Processing, ICPP 2023 - Workshops Proceedings |
Publisher | Association for Computing Machinery |
Pages | 172-179 |
Number of pages | 8 |
ISBN (Electronic) | 9798400708435 |
DOIs | |
State | Published - Aug 7 2023 |
Event | 52nd International Conference on Parallel Processing, ICPP 2023 - Workshops Proceedings - Salt Lake City, United States Duration: Aug 7 2023 → Aug 10 2023 |
Publication series
Name | ACM International Conference Proceeding Series |
---|
Conference
Conference | 52nd International Conference on Parallel Processing, ICPP 2023 - Workshops Proceedings |
---|---|
Country/Territory | United States |
City | Salt Lake City |
Period | 08/7/23 → 08/10/23 |
Funding
Thanks to Steven Young and Guojing Cong (ORNL), Travis Johnston (Striveworks), and Josh Romero (NVIDIA) for insightful feedback and assistance. This work was supported by the Office of Materials and Chemical Technologies within the Office of Nuclear Energy, U.S. Department of Energy, and used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725 This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).
Keywords
- evolutionary computation
- high performance computing
- hyperparameter optimization
- machine learning
- molecular simulation
- multiobjective optimization
- neural network potentials
- neural networks