Accelerating NWChem coupled cluster through dataflow-based execution

Heike Jagode; Anthony Danalis; George Bosilca; Jack Dongarra

doi:10.1007/978-3-319-32149-3_35

Accelerating NWChem coupled cluster through dataflow-based execution

Heike Jagode, Anthony Danalis, George Bosilca, Jack Dongarra

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

Numerical techniques used for describing many-body systems, such as the Coupled Cluster methods (CC) of the quantum chemistry package NWChem, are of extreme interest to the computational chemistry community in fields such as catalytic reactions, solar energy, and bio-mass conversion. In spite of their importance, many of these computationally intensive algorithms have traditionally been thought of in a fairly linear fashion, or are parallelised in coarse chunks. In this paper, we present our effort of converting the NWChem’s CC code into a dataflow-based form that is capable of utilizing the task scheduling system PaRSEC (Parallel Runtime Scheduling and Execution Controller) – a software package designed to enable high performance computing at scale. We discuss the modularity of our approach and explain how the PaRSEC-enabled dataflow version of the subroutines seamlessly integrate into the NWChem codebase. Furthermore, we argue how the CC algorithms can be easily decomposed into finer grained tasks (compared to the original version of NWChem); and how data distribution and load balancing are decoupled and can be tuned independently. We demonstrate performance acceleration by more than a factor of two in the execution of the entire CC component of NWChem, concluding that the utilization of dataflow-based execution for CC methods enables more efficient and scalable computation.

Original language	English
Title of host publication	Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers
Editors	Ewa Deelman, Jack Dongarra, Konrad Karczewski, Roman Wyrzykowski, Jacek Kitowski, Kazimierz Wiatr
Publisher	Springer Verlag
Pages	366-376
Number of pages	11
ISBN (Print)	9783319321486
DOIs	https://doi.org/10.1007/978-3-319-32149-3_35
State	Published - 2016
Event	11th International Conference on Parallel Processing and Applied Mathematics, PPAM 2015 - Krakow, Poland Duration: Sep 6 2015 → Sep 9 2015

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9573
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	11th International Conference on Parallel Processing and Applied Mathematics, PPAM 2015
Country/Territory	Poland
City	Krakow
Period	09/6/15 → 09/9/15

Funding

This material is based upon work supported in part by the Air Force Office of Scientific Research under AFOSR Award No. FA9550-12-1-0476, and the DOE Office of Science, Advanced Scientific Computing Research, under award No. DE-SC0006733 “SUPER - Institute for Sustained Performance, Energy and Resilience”. A portion of this research was performed using EMSL, a DOE Office of Science User Facility sponsored by the Office of Biological and Environmental Research and located at Pacific Northwest National Laboratory.

Keywords

CCSD
DAG
Dataflow
NWChem
PTG
PaRSEC
Tasks

Access to Document

10.1007/978-3-319-32149-3_35

Cite this

Jagode, H., Danalis, A., Bosilca, G., & Dongarra, J. (2016). Accelerating NWChem coupled cluster through dataflow-based execution. In E. Deelman, J. Dongarra, K. Karczewski, R. Wyrzykowski, J. Kitowski, & K. Wiatr (Eds.), Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers (pp. 366-376). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9573). Springer Verlag. https://doi.org/10.1007/978-3-319-32149-3_35

Jagode, Heike ; Danalis, Anthony ; Bosilca, George et al. / Accelerating NWChem coupled cluster through dataflow-based execution. Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers. editor / Ewa Deelman ; Jack Dongarra ; Konrad Karczewski ; Roman Wyrzykowski ; Jacek Kitowski ; Kazimierz Wiatr. Springer Verlag, 2016. pp. 366-376 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d02ad40c8d0748369158f2994eac9eee,

title = "Accelerating NWChem coupled cluster through dataflow-based execution",

abstract = "Numerical techniques used for describing many-body systems, such as the Coupled Cluster methods (CC) of the quantum chemistry package NWChem, are of extreme interest to the computational chemistry community in fields such as catalytic reactions, solar energy, and bio-mass conversion. In spite of their importance, many of these computationally intensive algorithms have traditionally been thought of in a fairly linear fashion, or are parallelised in coarse chunks. In this paper, we present our effort of converting the NWChem{\textquoteright}s CC code into a dataflow-based form that is capable of utilizing the task scheduling system PaRSEC (Parallel Runtime Scheduling and Execution Controller) – a software package designed to enable high performance computing at scale. We discuss the modularity of our approach and explain how the PaRSEC-enabled dataflow version of the subroutines seamlessly integrate into the NWChem codebase. Furthermore, we argue how the CC algorithms can be easily decomposed into finer grained tasks (compared to the original version of NWChem); and how data distribution and load balancing are decoupled and can be tuned independently. We demonstrate performance acceleration by more than a factor of two in the execution of the entire CC component of NWChem, concluding that the utilization of dataflow-based execution for CC methods enables more efficient and scalable computation.",

keywords = "CCSD, DAG, Dataflow, NWChem, PTG, PaRSEC, Tasks",

author = "Heike Jagode and Anthony Danalis and George Bosilca and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 11th International Conference on Parallel Processing and Applied Mathematics, PPAM 2015 ; Conference date: 06-09-2015 Through 09-09-2015",

year = "2016",

doi = "10.1007/978-3-319-32149-3_35",

language = "English",

isbn = "9783319321486",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "366--376",

editor = "Ewa Deelman and Jack Dongarra and Konrad Karczewski and Roman Wyrzykowski and Jacek Kitowski and Kazimierz Wiatr",

booktitle = "Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers",

}

Jagode, H, Danalis, A, Bosilca, G & Dongarra, J 2016, Accelerating NWChem coupled cluster through dataflow-based execution. in E Deelman, J Dongarra, K Karczewski, R Wyrzykowski, J Kitowski & K Wiatr (eds), Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9573, Springer Verlag, pp. 366-376, 11th International Conference on Parallel Processing and Applied Mathematics, PPAM 2015, Krakow, Poland, 09/6/15. https://doi.org/10.1007/978-3-319-32149-3_35

Accelerating NWChem coupled cluster through dataflow-based execution. / Jagode, Heike; Danalis, Anthony; Bosilca, George et al.
Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers. ed. / Ewa Deelman; Jack Dongarra; Konrad Karczewski; Roman Wyrzykowski; Jacek Kitowski; Kazimierz Wiatr. Springer Verlag, 2016. p. 366-376 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9573).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Accelerating NWChem coupled cluster through dataflow-based execution

AU - Jagode, Heike

AU - Danalis, Anthony

AU - Bosilca, George

AU - Dongarra, Jack

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - Numerical techniques used for describing many-body systems, such as the Coupled Cluster methods (CC) of the quantum chemistry package NWChem, are of extreme interest to the computational chemistry community in fields such as catalytic reactions, solar energy, and bio-mass conversion. In spite of their importance, many of these computationally intensive algorithms have traditionally been thought of in a fairly linear fashion, or are parallelised in coarse chunks. In this paper, we present our effort of converting the NWChem’s CC code into a dataflow-based form that is capable of utilizing the task scheduling system PaRSEC (Parallel Runtime Scheduling and Execution Controller) – a software package designed to enable high performance computing at scale. We discuss the modularity of our approach and explain how the PaRSEC-enabled dataflow version of the subroutines seamlessly integrate into the NWChem codebase. Furthermore, we argue how the CC algorithms can be easily decomposed into finer grained tasks (compared to the original version of NWChem); and how data distribution and load balancing are decoupled and can be tuned independently. We demonstrate performance acceleration by more than a factor of two in the execution of the entire CC component of NWChem, concluding that the utilization of dataflow-based execution for CC methods enables more efficient and scalable computation.

AB - Numerical techniques used for describing many-body systems, such as the Coupled Cluster methods (CC) of the quantum chemistry package NWChem, are of extreme interest to the computational chemistry community in fields such as catalytic reactions, solar energy, and bio-mass conversion. In spite of their importance, many of these computationally intensive algorithms have traditionally been thought of in a fairly linear fashion, or are parallelised in coarse chunks. In this paper, we present our effort of converting the NWChem’s CC code into a dataflow-based form that is capable of utilizing the task scheduling system PaRSEC (Parallel Runtime Scheduling and Execution Controller) – a software package designed to enable high performance computing at scale. We discuss the modularity of our approach and explain how the PaRSEC-enabled dataflow version of the subroutines seamlessly integrate into the NWChem codebase. Furthermore, we argue how the CC algorithms can be easily decomposed into finer grained tasks (compared to the original version of NWChem); and how data distribution and load balancing are decoupled and can be tuned independently. We demonstrate performance acceleration by more than a factor of two in the execution of the entire CC component of NWChem, concluding that the utilization of dataflow-based execution for CC methods enables more efficient and scalable computation.

KW - CCSD

KW - DAG

KW - Dataflow

KW - NWChem

KW - PTG

KW - PaRSEC

KW - Tasks

UR - http://www.scopus.com/inward/record.url?scp=84968570374&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-32149-3_35

DO - 10.1007/978-3-319-32149-3_35

M3 - Conference contribution

AN - SCOPUS:84968570374

SN - 9783319321486

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 366

EP - 376

BT - Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers

A2 - Deelman, Ewa

A2 - Dongarra, Jack

A2 - Karczewski, Konrad

A2 - Wyrzykowski, Roman

A2 - Kitowski, Jacek

A2 - Wiatr, Kazimierz

PB - Springer Verlag

T2 - 11th International Conference on Parallel Processing and Applied Mathematics, PPAM 2015

Y2 - 6 September 2015 through 9 September 2015

ER -

Jagode H, Danalis A, Bosilca G, Dongarra J. Accelerating NWChem coupled cluster through dataflow-based execution. In Deelman E, Dongarra J, Karczewski K, Wyrzykowski R, Kitowski J, Wiatr K, editors, Parallel Processing and Applied Mathematics - 11th International Conference, PPAM 2015, Revised Selected Papers. Springer Verlag. 2016. p. 366-376. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-32149-3_35

Accelerating NWChem coupled cluster through dataflow-based execution

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this