Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List

Christopher M. Siefert; Carl Pearson; Stephen L. Olivier; Andrey Prokopenko; Jonathan Hu; Timothy J. Fuller

doi:10.1145/3624062.3624203

Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List

Christopher M. Siefert, Carl Pearson, Stephen L. Olivier, Andrey Prokopenko, Jonathan Hu, Timothy J. Fuller

Computational Coupled Physics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

As a rule, Top 500 class supercomputers are extensively benchmarked as part of their acceptance testing process. However, barring publicly posted LINPACK / HPCG results, most benchmark results are often inaccessible outside the hosting institution. Moreover, these higher level benchmarks do not provide easy answers to common questions such as "What is the realizable memory bandwidth?"or "What is the launch latency on the accelerator?"To partially address these issues, we executed selected single-node micro-benchmarks - focused on latencies and memory bandwidth - on every US Department of Energy system above rank 150 of the June 2023 Top 500 list, excepting NERSC's Cori and ORNL's Frontier TDS (now decommissioned or repurposed). We hope to provide an easy "first stop"reference for users of current Top 500 systems and inspire users and administrators of other Top 500 systems to similarly compile and make available benchmark results for their systems.

Original language	English
Title of host publication	Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023
Publisher	Association for Computing Machinery
Pages	1298-1305
Number of pages	8
ISBN (Electronic)	9798400707858
DOIs	https://doi.org/10.1145/3624062.3624203
State	Published - Nov 12 2023
Event	2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023 - Denver, United States Duration: Nov 12 2023 → Nov 17 2023

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023
Country/Territory	United States
City	Denver
Period	11/12/23 → 11/17/23

Funding

This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This research used resources of the National Energy Research Scientific Computing Center, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231. This research made use of Idaho National Laboratory computing resources which are supported by the Office of Nuclear Energy of the U.S. Department of Energy and the Nuclear Science User Facilities under Contract No. DE-AC07-05ID14517. This research used resources of the Argonne Leadership Computing Facility, a DOE Office of Science User Facility supported under Contract No. DE-AC02-06CH11357. This research used resources of the Los Alamos National Laboratory, supported by the US Department of Energy under contract No 89233218CNA000001. The authors would also like to thank Christopher Knight of Argonne National Laboratory and James Elliott of Sandia National Laboratories for consulting on configuration and run options on various machines. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA-0003525. This written work is authored by an employee of NTESS. The employee, not NTESS, owns the right, title and interest in and to the written work and is responsible for its contents. Any subjective views or opinions that might be expressed in the written work do not necessarily represent the views of the U.S. Government. This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the U.S. Department of Energy. The publisher acknowledges that the U.S. Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this written work or allow others to do so, for U.S. Government purposes. The DOE will provide public access to results of federally sponsored research in accordance with the DOE Public Access Plan.

Keywords

high performance computing
micro-benchmarking
supercomputing
top500

Access to Document

10.1145/3624062.3624203

Cite this

Siefert, C. M., Pearson, C., Olivier, S. L., Prokopenko, A., Hu, J., & Fuller, T. J. (2023). Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List. In Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023 (pp. 1298-1305). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3624062.3624203

Siefert, Christopher M. ; Pearson, Carl ; Olivier, Stephen L. et al. / Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List. Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023. Association for Computing Machinery, 2023. pp. 1298-1305 (ACM International Conference Proceeding Series).

@inproceedings{0e3c0f734e4241c69cff3b6fa4b8796b,

title = "Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List",

abstract = "As a rule, Top 500 class supercomputers are extensively benchmarked as part of their acceptance testing process. However, barring publicly posted LINPACK / HPCG results, most benchmark results are often inaccessible outside the hosting institution. Moreover, these higher level benchmarks do not provide easy answers to common questions such as {"}What is the realizable memory bandwidth?{"}or {"}What is the launch latency on the accelerator?{"}To partially address these issues, we executed selected single-node micro-benchmarks - focused on latencies and memory bandwidth - on every US Department of Energy system above rank 150 of the June 2023 Top 500 list, excepting NERSC's Cori and ORNL's Frontier TDS (now decommissioned or repurposed). We hope to provide an easy {"}first stop{"}reference for users of current Top 500 systems and inspire users and administrators of other Top 500 systems to similarly compile and make available benchmark results for their systems.",

keywords = "high performance computing, micro-benchmarking, supercomputing, top500",

author = "Siefert, {Christopher M.} and Carl Pearson and Olivier, {Stephen L.} and Andrey Prokopenko and Jonathan Hu and Fuller, {Timothy J.}",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.; 2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023 ; Conference date: 12-11-2023 Through 17-11-2023",

year = "2023",

month = nov,

day = "12",

doi = "10.1145/3624062.3624203",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "1298--1305",

booktitle = "Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023",

}

Siefert, CM, Pearson, C, Olivier, SL, Prokopenko, A, Hu, J & Fuller, TJ 2023, Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List. in Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 1298-1305, 2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023, Denver, United States, 11/12/23. https://doi.org/10.1145/3624062.3624203

Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List. / Siefert, Christopher M.; Pearson, Carl; Olivier, Stephen L. et al.
Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023. Association for Computing Machinery, 2023. p. 1298-1305 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List

AU - Siefert, Christopher M.

AU - Pearson, Carl

AU - Olivier, Stephen L.

AU - Prokopenko, Andrey

AU - Hu, Jonathan

AU - Fuller, Timothy J.

PY - 2023/11/12

Y1 - 2023/11/12

N2 - As a rule, Top 500 class supercomputers are extensively benchmarked as part of their acceptance testing process. However, barring publicly posted LINPACK / HPCG results, most benchmark results are often inaccessible outside the hosting institution. Moreover, these higher level benchmarks do not provide easy answers to common questions such as "What is the realizable memory bandwidth?"or "What is the launch latency on the accelerator?"To partially address these issues, we executed selected single-node micro-benchmarks - focused on latencies and memory bandwidth - on every US Department of Energy system above rank 150 of the June 2023 Top 500 list, excepting NERSC's Cori and ORNL's Frontier TDS (now decommissioned or repurposed). We hope to provide an easy "first stop"reference for users of current Top 500 systems and inspire users and administrators of other Top 500 systems to similarly compile and make available benchmark results for their systems.

AB - As a rule, Top 500 class supercomputers are extensively benchmarked as part of their acceptance testing process. However, barring publicly posted LINPACK / HPCG results, most benchmark results are often inaccessible outside the hosting institution. Moreover, these higher level benchmarks do not provide easy answers to common questions such as "What is the realizable memory bandwidth?"or "What is the launch latency on the accelerator?"To partially address these issues, we executed selected single-node micro-benchmarks - focused on latencies and memory bandwidth - on every US Department of Energy system above rank 150 of the June 2023 Top 500 list, excepting NERSC's Cori and ORNL's Frontier TDS (now decommissioned or repurposed). We hope to provide an easy "first stop"reference for users of current Top 500 systems and inspire users and administrators of other Top 500 systems to similarly compile and make available benchmark results for their systems.

KW - high performance computing

KW - micro-benchmarking

KW - supercomputing

KW - top500

UR - http://www.scopus.com/inward/record.url?scp=85178093353&partnerID=8YFLogxK

U2 - 10.1145/3624062.3624203

DO - 10.1145/3624062.3624203

M3 - Conference contribution

AN - SCOPUS:85178093353

T3 - ACM International Conference Proceeding Series

SP - 1298

EP - 1305

BT - Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023

PB - Association for Computing Machinery

T2 - 2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023

Y2 - 12 November 2023 through 17 November 2023

ER -

Siefert CM, Pearson C, Olivier SL, Prokopenko A, Hu J, Fuller TJ. Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List. In Proceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023. Association for Computing Machinery. 2023. p. 1298-1305. (ACM International Conference Proceeding Series). doi: 10.1145/3624062.3624203

Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this