Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer

John Ossyra; Ada Sedova; Arnold Tharrington; Frank Noé; Cecilia Clementi; Jeremy C. Smith

doi:10.1007/978-3-030-34356-9_30

Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer

John Ossyra, Ada Sedova, Arnold Tharrington, Frank Noé, Cecilia Clementi, Jeremy C. Smith

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Scopus citations

Abstract

Molecular dynamics (MD) simulations must take very small (femtosecond) integration steps in simulation-time to avoid numerical errors. Efficient use of parallel programming models and accelerators in state-of-the art MD programs now is pushing Moore’s limit for time-per-MD step. As a result, directly simulating timescales beyond milliseconds will not be attainable directly, even at exascale. However, concepts from statistical physics can be used to combine many parallel simulations to provide information about longer timescales and to adequately sample the simulation space, while preserving details about the dynamics of the system. Implementing such an approach requires a workflow program that allows adaptable steering of task assignments based on extensive statistical analysis of intermediate results. Here we report the implementation of such an adaptable workflow program to drive simulations on the Summit IBM Power System AC922, a pre-exascale supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). We compare to experiences on Titan, Summit’s predecessor, report the performance of the workflow and its components, and describe the porting process. We find that using a workflow program managed by a Mongo database can provide the fault tolerance, scalable performance, task dispatch rate, and reconfigurability required for robust and portable implementation of ensemble simulations such as are used in enhanced-sampling molecular dynamics. This type of workflow generator can also be used to provide adaptive steering of ensemble simulations for other applications in addition to MD.

Original language	English
Title of host publication	High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers
Editors	Michèle Weiland, Guido Juckeland, Sadaf Alam, Heike Jagode
Publisher	Springer
Pages	397-417
Number of pages	21
ISBN (Print)	9783030343552
DOIs	https://doi.org/10.1007/978-3-030-34356-9_30
State	Published - 2019
Event	34th International Conference on High Performance Computing, ISC High Performance 2019 - Frankfurt, Germany Duration: Jun 16 2019 → Jun 20 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11887 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	34th International Conference on High Performance Computing, ISC High Performance 2019
Country/Territory	Germany
City	Frankfurt
Period	06/16/19 → 06/20/19

Funding

The authors would like to acknowledge Micholas Dean Smith for help with MD test systems preparation, and Shantenu Jha and lab for extensive help with incorporation of the Radical Cybertools software stack. An award of computer time was provided by the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR227525. ORNL is managed by UT-Battelle, LLC for the US Department of Energy. FN acknowledges European Commission (ERC CoG 772230) and Deutsche Forschungsgemeinschaft (NO 825/3-1). JCS acknowledges DOE contract ERKP752. CC acknowledges support from the National Science Foundation (CHE-1265929, CHE-1740990, CHE-1900374, and PHY-1427654) and the Welch Foundation (C-1570).

Funders	Funder number
DOE Office of Science	DE-AC05-00OR227525
Shantenu Jha
US Department of Energy
National Science Foundation	CHE-1900374, PHY-1427654, 1900374, CHE-1740990, CHE-1265929
U.S. Department of Energy	ERKP752
Welch Foundation	C-1570
Oak Ridge National Laboratory
European Commission
European Research Council	772230
Deutsche Forschungsgemeinschaft	NO 825/3-1

Keywords

Adaptive sampling
High Performance Computing
Molecular dynamics
Scientific workflows

Access to Document

10.1007/978-3-030-34356-9_30

Cite this

Ossyra, J., Sedova, A., Tharrington, A., Noé, F., Clementi, C., & Smith, J. C. (2019). Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer. In M. Weiland, G. Juckeland, S. Alam, & H. Jagode (Eds.), High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers (pp. 397-417). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11887 LNCS). Springer. https://doi.org/10.1007/978-3-030-34356-9_30

Ossyra, John ; Sedova, Ada ; Tharrington, Arnold et al. / Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer. High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. editor / Michèle Weiland ; Guido Juckeland ; Sadaf Alam ; Heike Jagode. Springer, 2019. pp. 397-417 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ffc59064dd844014a05358c9306d9f79,

title = "Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer",

abstract = "Molecular dynamics (MD) simulations must take very small (femtosecond) integration steps in simulation-time to avoid numerical errors. Efficient use of parallel programming models and accelerators in state-of-the art MD programs now is pushing Moore{\textquoteright}s limit for time-per-MD step. As a result, directly simulating timescales beyond milliseconds will not be attainable directly, even at exascale. However, concepts from statistical physics can be used to combine many parallel simulations to provide information about longer timescales and to adequately sample the simulation space, while preserving details about the dynamics of the system. Implementing such an approach requires a workflow program that allows adaptable steering of task assignments based on extensive statistical analysis of intermediate results. Here we report the implementation of such an adaptable workflow program to drive simulations on the Summit IBM Power System AC922, a pre-exascale supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). We compare to experiences on Titan, Summit{\textquoteright}s predecessor, report the performance of the workflow and its components, and describe the porting process. We find that using a workflow program managed by a Mongo database can provide the fault tolerance, scalable performance, task dispatch rate, and reconfigurability required for robust and portable implementation of ensemble simulations such as are used in enhanced-sampling molecular dynamics. This type of workflow generator can also be used to provide adaptive steering of ensemble simulations for other applications in addition to MD.",

keywords = "Adaptive sampling, High Performance Computing, Molecular dynamics, Scientific workflows",

author = "John Ossyra and Ada Sedova and Arnold Tharrington and Frank No{\'e} and Cecilia Clementi and Smith, {Jeremy C.}",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 34th International Conference on High Performance Computing, ISC High Performance 2019 ; Conference date: 16-06-2019 Through 20-06-2019",

year = "2019",

doi = "10.1007/978-3-030-34356-9_30",

language = "English",

isbn = "9783030343552",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "397--417",

editor = "Mich{\`e}le Weiland and Guido Juckeland and Sadaf Alam and Heike Jagode",

booktitle = "High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers",

}

Ossyra, J, Sedova, A , Tharrington, A, Noé, F, Clementi, C & Smith, JC 2019, Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer. in M Weiland, G Juckeland, S Alam & H Jagode (eds), High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11887 LNCS, Springer, pp. 397-417, 34th International Conference on High Performance Computing, ISC High Performance 2019, Frankfurt, Germany, 06/16/19. https://doi.org/10.1007/978-3-030-34356-9_30

Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer. / Ossyra, John; Sedova, Ada ; Tharrington, Arnold et al.
High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. ed. / Michèle Weiland; Guido Juckeland; Sadaf Alam; Heike Jagode. Springer, 2019. p. 397-417 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11887 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer

AU - Ossyra, John

AU - Sedova, Ada

AU - Tharrington, Arnold

AU - Noé, Frank

AU - Clementi, Cecilia

AU - Smith, Jeremy C.

PY - 2019

Y1 - 2019

N2 - Molecular dynamics (MD) simulations must take very small (femtosecond) integration steps in simulation-time to avoid numerical errors. Efficient use of parallel programming models and accelerators in state-of-the art MD programs now is pushing Moore’s limit for time-per-MD step. As a result, directly simulating timescales beyond milliseconds will not be attainable directly, even at exascale. However, concepts from statistical physics can be used to combine many parallel simulations to provide information about longer timescales and to adequately sample the simulation space, while preserving details about the dynamics of the system. Implementing such an approach requires a workflow program that allows adaptable steering of task assignments based on extensive statistical analysis of intermediate results. Here we report the implementation of such an adaptable workflow program to drive simulations on the Summit IBM Power System AC922, a pre-exascale supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). We compare to experiences on Titan, Summit’s predecessor, report the performance of the workflow and its components, and describe the porting process. We find that using a workflow program managed by a Mongo database can provide the fault tolerance, scalable performance, task dispatch rate, and reconfigurability required for robust and portable implementation of ensemble simulations such as are used in enhanced-sampling molecular dynamics. This type of workflow generator can also be used to provide adaptive steering of ensemble simulations for other applications in addition to MD.

AB - Molecular dynamics (MD) simulations must take very small (femtosecond) integration steps in simulation-time to avoid numerical errors. Efficient use of parallel programming models and accelerators in state-of-the art MD programs now is pushing Moore’s limit for time-per-MD step. As a result, directly simulating timescales beyond milliseconds will not be attainable directly, even at exascale. However, concepts from statistical physics can be used to combine many parallel simulations to provide information about longer timescales and to adequately sample the simulation space, while preserving details about the dynamics of the system. Implementing such an approach requires a workflow program that allows adaptable steering of task assignments based on extensive statistical analysis of intermediate results. Here we report the implementation of such an adaptable workflow program to drive simulations on the Summit IBM Power System AC922, a pre-exascale supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). We compare to experiences on Titan, Summit’s predecessor, report the performance of the workflow and its components, and describe the porting process. We find that using a workflow program managed by a Mongo database can provide the fault tolerance, scalable performance, task dispatch rate, and reconfigurability required for robust and portable implementation of ensemble simulations such as are used in enhanced-sampling molecular dynamics. This type of workflow generator can also be used to provide adaptive steering of ensemble simulations for other applications in addition to MD.

KW - Adaptive sampling

KW - High Performance Computing

KW - Molecular dynamics

KW - Scientific workflows

UR - http://www.scopus.com/inward/record.url?scp=85076857448&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-34356-9_30

DO - 10.1007/978-3-030-34356-9_30

M3 - Conference contribution

AN - SCOPUS:85076857448

SN - 9783030343552

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 397

EP - 417

BT - High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers

A2 - Weiland, Michèle

A2 - Juckeland, Guido

A2 - Alam, Sadaf

A2 - Jagode, Heike

PB - Springer

T2 - 34th International Conference on High Performance Computing, ISC High Performance 2019

Y2 - 16 June 2019 through 20 June 2019

ER -

Ossyra J, Sedova A , Tharrington A, Noé F, Clementi C, Smith JC. Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer. In Weiland M, Juckeland G, Alam S, Jagode H, editors, High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. Springer. 2019. p. 397-417. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-34356-9_30

Porting Adaptive Ensemble Molecular Dynamics Workflows to the Summit Supercomputer

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this