A Sparse Distributed Gigascale Resolution Material Point Method

Yuxing Qiu; Samuel Temple Reeve; Minchen Li; Yin Yang; Stuart Ryan Slattery; Chenfanfu Jiang

doi:10.1145/3570160

A Sparse Distributed Gigascale Resolution Material Point Method

Yuxing Qiu, Samuel Temple Reeve, Minchen Li, Yin Yang, Stuart Ryan Slattery, Chenfanfu Jiang

Scalable Engineering Applications

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

In this article, we present a four-layer distributed simulation system and its adaptation to the Material Point Method (MPM). The system is built upon a performance portable C++ programming model targeting major High-Performance-Computing (HPC) platforms. A key ingredient of our system is a hierarchical block-tile-cell sparse grid data structure that is distributable to an arbitrary number of Message Passing Interface (MPI) ranks. We additionally propose strategies for efficient dynamic load balance optimization to maximize the efficiency of MPI tasks. Our simulation pipeline can easily switch among backend programming models, including OpenMP and CUDA, and can be effortlessly dispatched onto supercomputers and the cloud. Finally, we construct benchmark experiments and ablation studies on supercomputers and consumer workstations in a local network to evaluate the scalability and load balancing criteria. We demonstrate massively parallel, highly scalable, and gigascale resolution MPM simulations of up to 1.01 billion particles for less than 323.25 seconds per frame with 8 OpenSSH-connected workstations.

Original language	English
Article number	22
Journal	ACM Transactions on Graphics
Volume	42
Issue number	2
DOIs	https://doi.org/10.1145/3570160
State	Published - Jan 16 2023

Funding

This work has been supported in part by NSF CAREER 2153851, CCF2153863, ECCS-2023780, DOE ORNL contract 4000171342, NSF 2244651 and 2301040. Additionally, this work was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. DOE Office of Science and the NNSA. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy (DOE). The publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan.

Funders	Funder number
Oak Ridge National Laboratory	17-SC-20-SC, 2301040, 4000171342, 2244651
National Science Foundation	2153851, ECCS-2023780, CCF2153863
U.S. Department of Energy	DE-AC05-00OR22725
Office of Science
National Nuclear Security Administration

Keywords

High Performance Computing
Material Point Method
distributed system and computing

Access to Document

10.1145/3570160

Cite this

@article{3b71b9eeb7ba4ecf983d97747c621b58,

title = "A Sparse Distributed Gigascale Resolution Material Point Method",

abstract = "In this article, we present a four-layer distributed simulation system and its adaptation to the Material Point Method (MPM). The system is built upon a performance portable C++ programming model targeting major High-Performance-Computing (HPC) platforms. A key ingredient of our system is a hierarchical block-tile-cell sparse grid data structure that is distributable to an arbitrary number of Message Passing Interface (MPI) ranks. We additionally propose strategies for efficient dynamic load balance optimization to maximize the efficiency of MPI tasks. Our simulation pipeline can easily switch among backend programming models, including OpenMP and CUDA, and can be effortlessly dispatched onto supercomputers and the cloud. Finally, we construct benchmark experiments and ablation studies on supercomputers and consumer workstations in a local network to evaluate the scalability and load balancing criteria. We demonstrate massively parallel, highly scalable, and gigascale resolution MPM simulations of up to 1.01 billion particles for less than 323.25 seconds per frame with 8 OpenSSH-connected workstations.",

keywords = "High Performance Computing, Material Point Method, distributed system and computing",

author = "Yuxing Qiu and Reeve, {Samuel Temple} and Minchen Li and Yin Yang and Slattery, {Stuart Ryan} and Chenfanfu Jiang",

note = "Publisher Copyright: {\textcopyright} 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.",

year = "2023",

month = jan,

day = "16",

doi = "10.1145/3570160",

language = "English",

volume = "42",

journal = "ACM Transactions on Graphics",

issn = "0730-0301",

publisher = "Association for Computing Machinery (ACM)",

number = "2",

}

TY - JOUR

T1 - A Sparse Distributed Gigascale Resolution Material Point Method

AU - Qiu, Yuxing

AU - Reeve, Samuel Temple

AU - Li, Minchen

AU - Yang, Yin

AU - Slattery, Stuart Ryan

AU - Jiang, Chenfanfu

PY - 2023/1/16

Y1 - 2023/1/16

N2 - In this article, we present a four-layer distributed simulation system and its adaptation to the Material Point Method (MPM). The system is built upon a performance portable C++ programming model targeting major High-Performance-Computing (HPC) platforms. A key ingredient of our system is a hierarchical block-tile-cell sparse grid data structure that is distributable to an arbitrary number of Message Passing Interface (MPI) ranks. We additionally propose strategies for efficient dynamic load balance optimization to maximize the efficiency of MPI tasks. Our simulation pipeline can easily switch among backend programming models, including OpenMP and CUDA, and can be effortlessly dispatched onto supercomputers and the cloud. Finally, we construct benchmark experiments and ablation studies on supercomputers and consumer workstations in a local network to evaluate the scalability and load balancing criteria. We demonstrate massively parallel, highly scalable, and gigascale resolution MPM simulations of up to 1.01 billion particles for less than 323.25 seconds per frame with 8 OpenSSH-connected workstations.

AB - In this article, we present a four-layer distributed simulation system and its adaptation to the Material Point Method (MPM). The system is built upon a performance portable C++ programming model targeting major High-Performance-Computing (HPC) platforms. A key ingredient of our system is a hierarchical block-tile-cell sparse grid data structure that is distributable to an arbitrary number of Message Passing Interface (MPI) ranks. We additionally propose strategies for efficient dynamic load balance optimization to maximize the efficiency of MPI tasks. Our simulation pipeline can easily switch among backend programming models, including OpenMP and CUDA, and can be effortlessly dispatched onto supercomputers and the cloud. Finally, we construct benchmark experiments and ablation studies on supercomputers and consumer workstations in a local network to evaluate the scalability and load balancing criteria. We demonstrate massively parallel, highly scalable, and gigascale resolution MPM simulations of up to 1.01 billion particles for less than 323.25 seconds per frame with 8 OpenSSH-connected workstations.

KW - High Performance Computing

KW - Material Point Method

KW - distributed system and computing

UR - http://www.scopus.com/inward/record.url?scp=85153854543&partnerID=8YFLogxK

U2 - 10.1145/3570160

DO - 10.1145/3570160

M3 - Article

AN - SCOPUS:85153854543

SN - 0730-0301

VL - 42

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

IS - 2

M1 - 22

ER -

A Sparse Distributed Gigascale Resolution Material Point Method

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this