Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale

Hao Lu; Michael Matheson; Vladyslav Oles; Austin Ellis; Wayne Joubert; Feiyi Wang

doi:10.1109/SC41404.2022.00083

Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale

Hao Lu, Michael Matheson, Vladyslav Oles, Austin Ellis, Wayne Joubert, Feiyi Wang

Analytics and AI Methods at Scale

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

The rise of machine learning (ML) applications and their use of mixed precision to perform interesting science are driving forces behind AI for science on HPC. The convergence of ML and HPC with mixed precision offers the possibility of transformational changes in computational science. The HPL-AI benchmark is designed to measure the performance of mixed precision arithmetic as opposed to the HPL benchmark which measures double precision performance. Pushing the limits of systems at extreme scale is nontrivial -little public literature explores optimization of mixed precision computations at this scale. In this work, we demonstrate how to scale up the HPL-AI benchmark on the pre-exascale Summit and exascale Frontier systems at the Oak Ridge Leadership Computing Facility (OLCF) with a cross-platform design. We present the implementation, performance results, and a guideline of optimization strategies employed for delivering portable performance on both AMD and NVIDIA GPUs at extreme scale.

Original language	English
Title of host publication	Proceedings of SC 2022
Subtitle of host publication	International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	IEEE Computer Society
ISBN (Electronic)	9781665454445
DOIs	https://doi.org/10.1109/SC41404.2022.00083
State	Published - 2022
Event	2022 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2022 - Dallas, United States Duration: Nov 13 2022 → Nov 18 2022

Publication series

Name	International Conference for High Performance Computing, Networking, Storage and Analysis, SC
Volume	2022-November
ISSN (Print)	2167-4329
ISSN (Electronic)	2167-4337

Conference

Conference	2022 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2022
Country/Territory	United States
City	Dallas
Period	11/13/22 → 11/18/22

Funding

This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This manuscript has been authored in part by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

Exascale computing
High performance computing
Linear algebra
Parallel programming

Access to Document

10.1109/SC41404.2022.00083

Cite this

Lu, H., Matheson, M., Oles, V., Ellis, A., Joubert, W., & Wang, F. (2022). Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale. In Proceedings of SC 2022: International Conference for High Performance Computing, Networking, Storage and Analysis (International Conference for High Performance Computing, Networking, Storage and Analysis, SC; Vol. 2022-November). IEEE Computer Society. https://doi.org/10.1109/SC41404.2022.00083

Lu, Hao ; Matheson, Michael ; Oles, Vladyslav et al. / Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale. Proceedings of SC 2022: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, 2022. (International Conference for High Performance Computing, Networking, Storage and Analysis, SC).

@inproceedings{1ed3f347467f4334bed2a045c913d7ae,

title = "Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale",

abstract = "The rise of machine learning (ML) applications and their use of mixed precision to perform interesting science are driving forces behind AI for science on HPC. The convergence of ML and HPC with mixed precision offers the possibility of transformational changes in computational science. The HPL-AI benchmark is designed to measure the performance of mixed precision arithmetic as opposed to the HPL benchmark which measures double precision performance. Pushing the limits of systems at extreme scale is nontrivial -little public literature explores optimization of mixed precision computations at this scale. In this work, we demonstrate how to scale up the HPL-AI benchmark on the pre-exascale Summit and exascale Frontier systems at the Oak Ridge Leadership Computing Facility (OLCF) with a cross-platform design. We present the implementation, performance results, and a guideline of optimization strategies employed for delivering portable performance on both AMD and NVIDIA GPUs at extreme scale.",

keywords = "Exascale computing, High performance computing, Linear algebra, Parallel programming",

author = "Hao Lu and Michael Matheson and Vladyslav Oles and Austin Ellis and Wayne Joubert and Feiyi Wang",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2022 ; Conference date: 13-11-2022 Through 18-11-2022",

year = "2022",

doi = "10.1109/SC41404.2022.00083",

language = "English",

series = "International Conference for High Performance Computing, Networking, Storage and Analysis, SC",

publisher = "IEEE Computer Society",

booktitle = "Proceedings of SC 2022",

}

Lu, H, Matheson, M, Oles, V, Ellis, A, Joubert, W & Wang, F 2022, Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale. in Proceedings of SC 2022: International Conference for High Performance Computing, Networking, Storage and Analysis. International Conference for High Performance Computing, Networking, Storage and Analysis, SC, vol. 2022-November, IEEE Computer Society, 2022 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2022, Dallas, United States, 11/13/22. https://doi.org/10.1109/SC41404.2022.00083

Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale. / Lu, Hao; Matheson, Michael; Oles, Vladyslav et al.
Proceedings of SC 2022: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, 2022. (International Conference for High Performance Computing, Networking, Storage and Analysis, SC; Vol. 2022-November).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale

AU - Lu, Hao

AU - Matheson, Michael

AU - Oles, Vladyslav

AU - Ellis, Austin

AU - Joubert, Wayne

AU - Wang, Feiyi

PY - 2022

Y1 - 2022

N2 - The rise of machine learning (ML) applications and their use of mixed precision to perform interesting science are driving forces behind AI for science on HPC. The convergence of ML and HPC with mixed precision offers the possibility of transformational changes in computational science. The HPL-AI benchmark is designed to measure the performance of mixed precision arithmetic as opposed to the HPL benchmark which measures double precision performance. Pushing the limits of systems at extreme scale is nontrivial -little public literature explores optimization of mixed precision computations at this scale. In this work, we demonstrate how to scale up the HPL-AI benchmark on the pre-exascale Summit and exascale Frontier systems at the Oak Ridge Leadership Computing Facility (OLCF) with a cross-platform design. We present the implementation, performance results, and a guideline of optimization strategies employed for delivering portable performance on both AMD and NVIDIA GPUs at extreme scale.

AB - The rise of machine learning (ML) applications and their use of mixed precision to perform interesting science are driving forces behind AI for science on HPC. The convergence of ML and HPC with mixed precision offers the possibility of transformational changes in computational science. The HPL-AI benchmark is designed to measure the performance of mixed precision arithmetic as opposed to the HPL benchmark which measures double precision performance. Pushing the limits of systems at extreme scale is nontrivial -little public literature explores optimization of mixed precision computations at this scale. In this work, we demonstrate how to scale up the HPL-AI benchmark on the pre-exascale Summit and exascale Frontier systems at the Oak Ridge Leadership Computing Facility (OLCF) with a cross-platform design. We present the implementation, performance results, and a guideline of optimization strategies employed for delivering portable performance on both AMD and NVIDIA GPUs at extreme scale.

KW - Exascale computing

KW - High performance computing

KW - Linear algebra

KW - Parallel programming

UR - http://www.scopus.com/inward/record.url?scp=85149311386&partnerID=8YFLogxK

U2 - 10.1109/SC41404.2022.00083

DO - 10.1109/SC41404.2022.00083

M3 - Conference contribution

AN - SCOPUS:85149311386

T3 - International Conference for High Performance Computing, Networking, Storage and Analysis, SC

BT - Proceedings of SC 2022

PB - IEEE Computer Society

T2 - 2022 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2022

Y2 - 13 November 2022 through 18 November 2022

ER -

Lu H, Matheson M, Oles V, Ellis A, Joubert W, Wang F. Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale. In Proceedings of SC 2022: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society. 2022. (International Conference for High Performance Computing, Networking, Storage and Analysis, SC). doi: 10.1109/SC41404.2022.00083

Climbing the Summit and Pushing the Frontier of Mixed Precision Benchmarks at Extreme Scale

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this