Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size

David Eberius; Philip Roth; David M. Rogers

doi:10.1109/P3HPC56579.2022.00008

Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size

David Eberius, Philip Roth, David M. Rogers

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

The roofline model provides a concise overview of the maximum performance capabilities of a given computer system through a combination of peak memory bandwidth and compute performance rates. The increasing complexity of scheduling and cache in recent GPUs, however, has introduced complicated performance variability that is not captured by arithmetic intensity alone. This work examines the effect of problem size and GPU launch configurations on roofline performance for V100, A100, MI100, and MI250X graphics processing units. We introduce an extended roofline model that takes problem size into account, and find that strong scaling on GPUs can be characterized by saturation problem sizes as additional key metrics. Saturation problem sizes break up a plot of GPU performance vs. problem size into three distinct performance regimes- size-limited, cache-bound, and DRAM-bound. With our extended roofline model, we are able to provide a robust view of these performance regimes across recent GPU architectures.

Original language	English
Title of host publication	Proceedings of P3HPC 2022
Subtitle of host publication	2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	26-35
Number of pages	10
ISBN (Electronic)	9781665460217
DOIs	https://doi.org/10.1109/P3HPC56579.2022.00008
State	Published - 2022
Event	5th IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC, P3HPC 2022 - Dallas, United States Duration: Nov 13 2022 → Nov 18 2022

Publication series

Name	Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	5th IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC, P3HPC 2022
Country/Territory	United States
City	Dallas
Period	11/13/22 → 11/18/22

Funding

ACKNOWLEDGMENTS This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Officeof Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

CUDA
GPU
HIP
HPC
performance analysis
roofline model

Access to Document

10.1109/P3HPC56579.2022.00008

Cite this

Eberius, D., Roth, P., & Rogers, D. M. (2022). Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size. In Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 26-35). (Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/P3HPC56579.2022.00008

Eberius, David ; Roth, Philip ; Rogers, David M. / Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size. Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 26-35 (Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{012240aac4ae46d5b0088911f8a5974f,

title = "Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size",

abstract = "The roofline model provides a concise overview of the maximum performance capabilities of a given computer system through a combination of peak memory bandwidth and compute performance rates. The increasing complexity of scheduling and cache in recent GPUs, however, has introduced complicated performance variability that is not captured by arithmetic intensity alone. This work examines the effect of problem size and GPU launch configurations on roofline performance for V100, A100, MI100, and MI250X graphics processing units. We introduce an extended roofline model that takes problem size into account, and find that strong scaling on GPUs can be characterized by saturation problem sizes as additional key metrics. Saturation problem sizes break up a plot of GPU performance vs. problem size into three distinct performance regimes- size-limited, cache-bound, and DRAM-bound. With our extended roofline model, we are able to provide a robust view of these performance regimes across recent GPU architectures.",

keywords = "CUDA, GPU, HIP, HPC, performance analysis, roofline model",

author = "David Eberius and Philip Roth and Rogers, {David M.}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 5th IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC, P3HPC 2022 ; Conference date: 13-11-2022 Through 18-11-2022",

year = "2022",

doi = "10.1109/P3HPC56579.2022.00008",

language = "English",

series = "Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "26--35",

booktitle = "Proceedings of P3HPC 2022",

}

Eberius, D, Roth, P & Rogers, DM 2022, Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size. in Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 26-35, 5th IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC, P3HPC 2022, Dallas, United States, 11/13/22. https://doi.org/10.1109/P3HPC56579.2022.00008

Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size. / Eberius, David; Roth, Philip ; Rogers, David M.
Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2022. p. 26-35 (Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size

AU - Eberius, David

AU - Roth, Philip

AU - Rogers, David M.

PY - 2022

Y1 - 2022

N2 - The roofline model provides a concise overview of the maximum performance capabilities of a given computer system through a combination of peak memory bandwidth and compute performance rates. The increasing complexity of scheduling and cache in recent GPUs, however, has introduced complicated performance variability that is not captured by arithmetic intensity alone. This work examines the effect of problem size and GPU launch configurations on roofline performance for V100, A100, MI100, and MI250X graphics processing units. We introduce an extended roofline model that takes problem size into account, and find that strong scaling on GPUs can be characterized by saturation problem sizes as additional key metrics. Saturation problem sizes break up a plot of GPU performance vs. problem size into three distinct performance regimes- size-limited, cache-bound, and DRAM-bound. With our extended roofline model, we are able to provide a robust view of these performance regimes across recent GPU architectures.

AB - The roofline model provides a concise overview of the maximum performance capabilities of a given computer system through a combination of peak memory bandwidth and compute performance rates. The increasing complexity of scheduling and cache in recent GPUs, however, has introduced complicated performance variability that is not captured by arithmetic intensity alone. This work examines the effect of problem size and GPU launch configurations on roofline performance for V100, A100, MI100, and MI250X graphics processing units. We introduce an extended roofline model that takes problem size into account, and find that strong scaling on GPUs can be characterized by saturation problem sizes as additional key metrics. Saturation problem sizes break up a plot of GPU performance vs. problem size into three distinct performance regimes- size-limited, cache-bound, and DRAM-bound. With our extended roofline model, we are able to provide a robust view of these performance regimes across recent GPU architectures.

KW - CUDA

KW - GPU

KW - HIP

KW - HPC

KW - performance analysis

KW - roofline model

UR - http://www.scopus.com/inward/record.url?scp=85147956077&partnerID=8YFLogxK

U2 - 10.1109/P3HPC56579.2022.00008

DO - 10.1109/P3HPC56579.2022.00008

M3 - Conference contribution

AN - SCOPUS:85147956077

T3 - Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 26

EP - 35

BT - Proceedings of P3HPC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 5th IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC, P3HPC 2022

Y2 - 13 November 2022 through 18 November 2022

ER -

Eberius D, Roth P , Rogers DM. Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size. In Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2022. p. 26-35. (Proceedings of P3HPC 2022: 2022 International Workshop on Performance, Portability and Productivity in HPC, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/P3HPC56579.2022.00008

Understanding Strong Scaling on GPUs Using Empirical Performance Saturation Size

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this