Parallel Computing for Module-Based Computational Experiment

Zhuo Yao; Dali Wang; Danial Riccuito; Fengming Yuan; Chunsheng Fang

doi:10.1007/978-3-030-22741-8_27

Parallel Computing for Module-Based Computational Experiment

Zhuo Yao, Dali Wang, Danial Riccuito, Fengming Yuan, Chunsheng Fang

Earth Systems Modeling

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Large-scale scientific code plays an important role in scientific researches. In order to facilitate module and element evaluation in scientific applications, we introduce a unit testing framework and describe the demand for module-based experiment customization. We then develop a parallel version of the unit testing framework to handle long-term simulations with a large amount of data. Specifically, we apply message passing based parallelization and I/O behavior optimization to improve the performance of the unit testing framework and use profiling result to guide the parallel process implementation. Finally, we present a case study on litter decomposition experiment using a standalone module from a large-scale Earth System Model. This case study is also a good demonstration on the scalability, portability, and high-efficiency of the framework.

Original language	English
Title of host publication	Computational Science – ICCS 2019 - 19th International Conference, Proceedings
Editors	João M.F. Rodrigues, Pedro J.S. Cardoso, Jânio Monteiro, Roberto Lam, Valeria V. Krzhizhanovskaya, Michael H. Lees, Peter M.A. Sloot, Jack J. Dongarra
Publisher	Springer Verlag
Pages	377-388
Number of pages	12
ISBN (Print)	9783030227401
DOIs	https://doi.org/10.1007/978-3-030-22741-8_27
State	Published - 2019
Event	19th International Conference on Computational Science, ICCS 2019 - Faro, Portugal Duration: Jun 12 2019 → Jun 14 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11537 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Computational Science, ICCS 2019
Country/Territory	Portugal
City	Faro
Period	06/12/19 → 06/14/19

Funding

Acknowledgement. Some part of this research is included in Yao’s Ph.D. dissertation (A Kernel Generation Framework for Scientific Legacy Code [12]) with the University of Tennessee, Knoxville. TN. This research was funded by the U.S. Department of Energy, Office of Science, Biological and Environmental Research program (E3SM and TES) and Advanced Scientific Computing Research program. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This research was funded by the U.S. Department of Energy, Office of Science, Biological and Environmental Research (Terrestrial Ecosystem Sciences), Advanced Scientific Computing Research (Interoperable Design of Extreme-scale Application Software).

Keywords

Message passing based parallelization
Parallel computing
Profiling
Scientific software

Access to Document

10.1007/978-3-030-22741-8_27

Cite this

Yao, Z., Wang, D., Riccuito, D., Yuan, F., & Fang, C. (2019). Parallel Computing for Module-Based Computational Experiment. In J. M. F. Rodrigues, P. J. S. Cardoso, J. Monteiro, R. Lam, V. V. Krzhizhanovskaya, M. H. Lees, P. M. A. Sloot, & J. J. Dongarra (Eds.), Computational Science – ICCS 2019 - 19th International Conference, Proceedings (pp. 377-388). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11537 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-22741-8_27

Yao, Zhuo ; Wang, Dali ; Riccuito, Danial et al. / Parallel Computing for Module-Based Computational Experiment. Computational Science – ICCS 2019 - 19th International Conference, Proceedings. editor / João M.F. Rodrigues ; Pedro J.S. Cardoso ; Jânio Monteiro ; Roberto Lam ; Valeria V. Krzhizhanovskaya ; Michael H. Lees ; Peter M.A. Sloot ; Jack J. Dongarra. Springer Verlag, 2019. pp. 377-388 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{94650c70c7f645aea5921eb49aaed2d2,

title = "Parallel Computing for Module-Based Computational Experiment",

abstract = "Large-scale scientific code plays an important role in scientific researches. In order to facilitate module and element evaluation in scientific applications, we introduce a unit testing framework and describe the demand for module-based experiment customization. We then develop a parallel version of the unit testing framework to handle long-term simulations with a large amount of data. Specifically, we apply message passing based parallelization and I/O behavior optimization to improve the performance of the unit testing framework and use profiling result to guide the parallel process implementation. Finally, we present a case study on litter decomposition experiment using a standalone module from a large-scale Earth System Model. This case study is also a good demonstration on the scalability, portability, and high-efficiency of the framework.",

keywords = "Message passing based parallelization, Parallel computing, Profiling, Scientific software",

author = "Zhuo Yao and Dali Wang and Danial Riccuito and Fengming Yuan and Chunsheng Fang",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 19th International Conference on Computational Science, ICCS 2019 ; Conference date: 12-06-2019 Through 14-06-2019",

year = "2019",

doi = "10.1007/978-3-030-22741-8_27",

language = "English",

isbn = "9783030227401",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "377--388",

editor = "Rodrigues, {Jo{\~a}o M.F.} and Cardoso, {Pedro J.S.} and J{\^a}nio Monteiro and Roberto Lam and Krzhizhanovskaya, {Valeria V.} and Lees, {Michael H.} and Sloot, {Peter M.A.} and Dongarra, {Jack J.}",

booktitle = "Computational Science – ICCS 2019 - 19th International Conference, Proceedings",

}

Yao, Z, Wang, D, Riccuito, D , Yuan, F & Fang, C 2019, Parallel Computing for Module-Based Computational Experiment. in JMF Rodrigues, PJS Cardoso, J Monteiro, R Lam, VV Krzhizhanovskaya, MH Lees, PMA Sloot & JJ Dongarra (eds), Computational Science – ICCS 2019 - 19th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11537 LNCS, Springer Verlag, pp. 377-388, 19th International Conference on Computational Science, ICCS 2019, Faro, Portugal, 06/12/19. https://doi.org/10.1007/978-3-030-22741-8_27

Parallel Computing for Module-Based Computational Experiment. / Yao, Zhuo; Wang, Dali; Riccuito, Danial et al.
Computational Science – ICCS 2019 - 19th International Conference, Proceedings. ed. / João M.F. Rodrigues; Pedro J.S. Cardoso; Jânio Monteiro; Roberto Lam; Valeria V. Krzhizhanovskaya; Michael H. Lees; Peter M.A. Sloot; Jack J. Dongarra. Springer Verlag, 2019. p. 377-388 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11537 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Parallel Computing for Module-Based Computational Experiment

AU - Yao, Zhuo

AU - Wang, Dali

AU - Riccuito, Danial

AU - Yuan, Fengming

AU - Fang, Chunsheng

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - Large-scale scientific code plays an important role in scientific researches. In order to facilitate module and element evaluation in scientific applications, we introduce a unit testing framework and describe the demand for module-based experiment customization. We then develop a parallel version of the unit testing framework to handle long-term simulations with a large amount of data. Specifically, we apply message passing based parallelization and I/O behavior optimization to improve the performance of the unit testing framework and use profiling result to guide the parallel process implementation. Finally, we present a case study on litter decomposition experiment using a standalone module from a large-scale Earth System Model. This case study is also a good demonstration on the scalability, portability, and high-efficiency of the framework.

AB - Large-scale scientific code plays an important role in scientific researches. In order to facilitate module and element evaluation in scientific applications, we introduce a unit testing framework and describe the demand for module-based experiment customization. We then develop a parallel version of the unit testing framework to handle long-term simulations with a large amount of data. Specifically, we apply message passing based parallelization and I/O behavior optimization to improve the performance of the unit testing framework and use profiling result to guide the parallel process implementation. Finally, we present a case study on litter decomposition experiment using a standalone module from a large-scale Earth System Model. This case study is also a good demonstration on the scalability, portability, and high-efficiency of the framework.

KW - Message passing based parallelization

KW - Parallel computing

KW - Profiling

KW - Scientific software

UR - http://www.scopus.com/inward/record.url?scp=85067701634&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-22741-8_27

DO - 10.1007/978-3-030-22741-8_27

M3 - Conference contribution

AN - SCOPUS:85067701634

SN - 9783030227401

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 377

EP - 388

BT - Computational Science – ICCS 2019 - 19th International Conference, Proceedings

A2 - Rodrigues, João M.F.

A2 - Cardoso, Pedro J.S.

A2 - Monteiro, Jânio

A2 - Lam, Roberto

A2 - Krzhizhanovskaya, Valeria V.

A2 - Lees, Michael H.

A2 - Sloot, Peter M.A.

A2 - Dongarra, Jack J.

PB - Springer Verlag

T2 - 19th International Conference on Computational Science, ICCS 2019

Y2 - 12 June 2019 through 14 June 2019

ER -

Yao Z, Wang D, Riccuito D , Yuan F, Fang C. Parallel Computing for Module-Based Computational Experiment. In Rodrigues JMF, Cardoso PJS, Monteiro J, Lam R, Krzhizhanovskaya VV, Lees MH, Sloot PMA, Dongarra JJ, editors, Computational Science – ICCS 2019 - 19th International Conference, Proceedings. Springer Verlag. 2019. p. 377-388. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-22741-8_27

Parallel Computing for Module-Based Computational Experiment

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this