Strong scaling of general-purpose molecular dynamics simulations on GPUs

Jens Glaser; Trung Dac Nguyen; Joshua A. Anderson; Pak Lui; Filippo Spiga; Jaime A. Millan; David C. Morse; Sharon C. Glotzer

doi:10.1016/j.cpc.2015.02.028

Strong scaling of general-purpose molecular dynamics simulations on GPUs

Jens Glaser, Trung Dac Nguyen, Joshua A. Anderson, Pak Lui, Filippo Spiga, Jaime A. Millan, David C. Morse, Sharon C. Glotzer

Research output: Contribution to journal › Article › peer-review

521 Scopus citations

Abstract

We describe a highly optimized implementation of MPI domain decomposition in a GPU-enabled, general-purpose molecular dynamics code, HOOMD-blue (Anderson and Glotzer, 2013). Our approach is inspired by a traditional CPU-based code, LAMMPS (Plimpton, 1995), but is implemented within a code that was designed for execution on GPUs from the start (Anderson et al.; 2008). The software supports short-ranged pair force and bond force fields and achieves optimal GPU performance using an autotuning algorithm. We are able to demonstrate equivalent or superior scaling on up to 3375 GPUs in Lennard-Jones and dissipative particle dynamics (DPD) simulations of up to 108 million particles. GPUDirect RDMA capabilities in recent GPU generations provide better performance in full double precision calculations. For a representative polymer physics application, HOOMD-blue 1.0 provides an effective GPU vs. CPU node speed-up of 12.5×.

Original language	English
Pages (from-to)	97-107
Number of pages	11
Journal	Computer Physics Communications
Volume	192
DOIs	https://doi.org/10.1016/j.cpc.2015.02.028
State	Published - Jul 1 2015
Externally published	Yes

Funding

This material is based upon work supported by the DOD / ASD (R&E) under Award No. N00244-09-1-0062 (JG, JAA, JAM, SCG). JG acknowledges support by DFG grant GL733/1-1 . We also acknowledge support by the National Science Foundation , Division of Materials Research , award DMR 1409620 (JAA and SCG), and award DMR 0907338 (JG and DCM). This work was partially supported by a Simons Investigator award from the Simons Foundation to Sharon Glotzer (SCG, JG). This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the US Department of Energy under Contract No. DE-AC05-00OR22725 . This research is part of the Blue Waters sustained-petascale computing project, which is supported by the National Science Foundation (award number ACI 1238993 ) and the state of Illinois . Blue Waters is a joint effort of the University of Illinois at Urbana–Champaign and its National Center for Supercomputing Applications. We thank the University of Cambridge for providing access to their Wilkes cluster. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the DOD/ASD(R&E). The Glotzer Group at the University of Michigan is a CUDA Research Center. Hardware support by NVIDIA is gratefully acknowledged.

Funders	Funder number
National Science Foundation	1238993, 1409620, 0907338, 1515306
U.S. Department of Defense
U.S. Department of Energy	DE-AC05-00OR22725, ACI 1238993
Division of Materials Research	DMR 0907338, DMR 1409620
Simons Foundation
Office of Science
NVIDIA
Astrophysics Science Division	N00244-09-1-0062
Deutsche Forschungsgemeinschaft	GL733/1-1

Keywords

Domain decomposition
LAMMPS
MPI/CUDA
Molecular dynamics
Multi-GPU
Strong scaling
Weak scaling

Access to Document

10.1016/j.cpc.2015.02.028

Cite this

@article{bc016f82a0304fc8b5587360e92d3b5e,

title = "Strong scaling of general-purpose molecular dynamics simulations on GPUs",

abstract = "We describe a highly optimized implementation of MPI domain decomposition in a GPU-enabled, general-purpose molecular dynamics code, HOOMD-blue (Anderson and Glotzer, 2013). Our approach is inspired by a traditional CPU-based code, LAMMPS (Plimpton, 1995), but is implemented within a code that was designed for execution on GPUs from the start (Anderson et al.; 2008). The software supports short-ranged pair force and bond force fields and achieves optimal GPU performance using an autotuning algorithm. We are able to demonstrate equivalent or superior scaling on up to 3375 GPUs in Lennard-Jones and dissipative particle dynamics (DPD) simulations of up to 108 million particles. GPUDirect RDMA capabilities in recent GPU generations provide better performance in full double precision calculations. For a representative polymer physics application, HOOMD-blue 1.0 provides an effective GPU vs. CPU node speed-up of 12.5×.",

keywords = "Domain decomposition, LAMMPS, MPI/CUDA, Molecular dynamics, Multi-GPU, Strong scaling, Weak scaling",

author = "Jens Glaser and Nguyen, {Trung Dac} and Anderson, {Joshua A.} and Pak Lui and Filippo Spiga and Millan, {Jaime A.} and Morse, {David C.} and Glotzer, {Sharon C.}",

year = "2015",

month = jul,

day = "1",

doi = "10.1016/j.cpc.2015.02.028",

language = "English",

volume = "192",

pages = "97--107",

journal = "Computer Physics Communications",

issn = "0010-4655",

publisher = "Elsevier",

}

TY - JOUR

T1 - Strong scaling of general-purpose molecular dynamics simulations on GPUs

AU - Glaser, Jens

AU - Nguyen, Trung Dac

AU - Anderson, Joshua A.

AU - Lui, Pak

AU - Spiga, Filippo

AU - Millan, Jaime A.

AU - Morse, David C.

AU - Glotzer, Sharon C.

PY - 2015/7/1

Y1 - 2015/7/1

N2 - We describe a highly optimized implementation of MPI domain decomposition in a GPU-enabled, general-purpose molecular dynamics code, HOOMD-blue (Anderson and Glotzer, 2013). Our approach is inspired by a traditional CPU-based code, LAMMPS (Plimpton, 1995), but is implemented within a code that was designed for execution on GPUs from the start (Anderson et al.; 2008). The software supports short-ranged pair force and bond force fields and achieves optimal GPU performance using an autotuning algorithm. We are able to demonstrate equivalent or superior scaling on up to 3375 GPUs in Lennard-Jones and dissipative particle dynamics (DPD) simulations of up to 108 million particles. GPUDirect RDMA capabilities in recent GPU generations provide better performance in full double precision calculations. For a representative polymer physics application, HOOMD-blue 1.0 provides an effective GPU vs. CPU node speed-up of 12.5×.

AB - We describe a highly optimized implementation of MPI domain decomposition in a GPU-enabled, general-purpose molecular dynamics code, HOOMD-blue (Anderson and Glotzer, 2013). Our approach is inspired by a traditional CPU-based code, LAMMPS (Plimpton, 1995), but is implemented within a code that was designed for execution on GPUs from the start (Anderson et al.; 2008). The software supports short-ranged pair force and bond force fields and achieves optimal GPU performance using an autotuning algorithm. We are able to demonstrate equivalent or superior scaling on up to 3375 GPUs in Lennard-Jones and dissipative particle dynamics (DPD) simulations of up to 108 million particles. GPUDirect RDMA capabilities in recent GPU generations provide better performance in full double precision calculations. For a representative polymer physics application, HOOMD-blue 1.0 provides an effective GPU vs. CPU node speed-up of 12.5×.

KW - Domain decomposition

KW - LAMMPS

KW - MPI/CUDA

KW - Molecular dynamics

KW - Multi-GPU

KW - Strong scaling

KW - Weak scaling

UR - http://www.scopus.com/inward/record.url?scp=84928049032&partnerID=8YFLogxK

U2 - 10.1016/j.cpc.2015.02.028

DO - 10.1016/j.cpc.2015.02.028

M3 - Article

AN - SCOPUS:84928049032

SN - 0010-4655

VL - 192

SP - 97

EP - 107

JO - Computer Physics Communications

JF - Computer Physics Communications

ER -

Strong scaling of general-purpose molecular dynamics simulations on GPUs

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this