PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications

Lingqi Zhang; Mohamed Wahib; Peng Chen; Jintao Meng; Xiao Wang; Toshio Endo; Satoshi Matsuoka

doi:10.1145/3577193.3593705

PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications

Lingqi Zhang
, Mohamed Wahib
, Peng Chen
, Jintao Meng
, Xiao Wang
, Toshio Endo
, Satoshi Matsuoka

Multiscale Biomedical Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

Iterative memory-bound solvers commonly occur in HPC codes. Typical GPU implementations have a loop on the host side that invokes the GPU kernel as much as time/algorithm steps there are. The termination of each kernel implicitly acts the barrier required after advancing the solution every time step. We propose an execution model for running memory-bound iterative GPU kernels: PERsistent KernelS (PERKS). In this model, the time loop is moved inside persistent kernel, and device-wide barriers are used for synchronization. We then reduce the traffic to device memory by caching subset of the output in each time step in the unused registers and shared memory. PERKS can be generalized to any iterative solver: they largely independent of the solver's implementation. We explain the design principle of PERKS and demonstrate effectiveness of PERKS for a wide range of iterative 2D/3D stencil benchmarks (geomean speedup of 2.12x for 2D stencils and 1.24x for 3D stencils over state-of-art libraries), and a Krylov subspace conjugate gradient solver (geomean speedup of 4.86x in smaller SpMV datasets from SuiteSparse and 1.43x in larger SpMV datasets over a state-of-art library). All PERKS-based implementations available at: https://github.com/neozhang307/PERKS.

Original language	English
Title of host publication	ACM ICS 2023 - Proceedings of the International Conference on Supercomputing
Publisher	Association for Computing Machinery
Pages	167-179
Number of pages	13
ISBN (Electronic)	9798400700569
DOIs	https://doi.org/10.1145/3577193.3593705
State	Published - Jun 21 2023
Event	37th ACM International Conference on Supercomputing, ICS 2023 - Orlando, United States Duration: Jun 21 2023 → Jun 23 2023

Publication series

Name	Proceedings of the International Conference on Supercomputing

Conference

Conference	37th ACM International Conference on Supercomputing, ICS 2023
Country/Territory	United States
City	Orlando
Period	06/21/23 → 06/23/23

Funding

This work was supported by JSPS KAKENHI under Grant Numbers JP22H03600 and JP21K17750. This work was supported by JST, PRESTO Grant Number JPMJPR20MA, Japan. This paper is based on results obtained from JPNP20006 project, commissioned by the New Energy and Industrial Technology Development Organization (NEDO). This manuscript has been co-authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The publisher acknowledges the US government license to provide public access under the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan/). The authors wish to express their sincere appreciation to Jens Domke, Aleksandr Drozd, Emil Vatai and other RIKEN R-CCS colleagues for their invaluable advice and guidance throughout the course of this research. They also wish to thank Dr. Zhao Tuowen from the SambaNova for the helpful discussions. Finally, the first author would also like to express his gratitude to RIKEN R-CCS for offering the opportunity to undertake this research in an intern program.

Keywords

GPU
iterative solvers
memory-bound
persistent kernel

Access to Document

10.1145/3577193.3593705

Cite this

Zhang, L., Wahib, M., Chen, P., Meng, J., Wang, X., Endo, T., & Matsuoka, S. (2023). PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications. In ACM ICS 2023 - Proceedings of the International Conference on Supercomputing (pp. 167-179). (Proceedings of the International Conference on Supercomputing). Association for Computing Machinery. https://doi.org/10.1145/3577193.3593705

@inproceedings{c0b842d198944433b23079679ffc3db2,

title = "PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications",

abstract = "Iterative memory-bound solvers commonly occur in HPC codes. Typical GPU implementations have a loop on the host side that invokes the GPU kernel as much as time/algorithm steps there are. The termination of each kernel implicitly acts the barrier required after advancing the solution every time step. We propose an execution model for running memory-bound iterative GPU kernels: PERsistent KernelS (PERKS). In this model, the time loop is moved inside persistent kernel, and device-wide barriers are used for synchronization. We then reduce the traffic to device memory by caching subset of the output in each time step in the unused registers and shared memory. PERKS can be generalized to any iterative solver: they largely independent of the solver's implementation. We explain the design principle of PERKS and demonstrate effectiveness of PERKS for a wide range of iterative 2D/3D stencil benchmarks (geomean speedup of 2.12x for 2D stencils and 1.24x for 3D stencils over state-of-art libraries), and a Krylov subspace conjugate gradient solver (geomean speedup of 4.86x in smaller SpMV datasets from SuiteSparse and 1.43x in larger SpMV datasets over a state-of-art library). All PERKS-based implementations available at: https://github.com/neozhang307/PERKS.",

keywords = "GPU, iterative solvers, memory-bound, persistent kernel",

author = "Lingqi Zhang and Mohamed Wahib and Peng Chen and Jintao Meng and Xiao Wang and Toshio Endo and Satoshi Matsuoka",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.; 37th ACM International Conference on Supercomputing, ICS 2023 ; Conference date: 21-06-2023 Through 23-06-2023",

year = "2023",

month = jun,

day = "21",

doi = "10.1145/3577193.3593705",

language = "English",

series = "Proceedings of the International Conference on Supercomputing",

publisher = "Association for Computing Machinery",

pages = "167--179",

booktitle = "ACM ICS 2023 - Proceedings of the International Conference on Supercomputing",

}

Zhang, L, Wahib, M, Chen, P, Meng, J, Wang, X, Endo, T & Matsuoka, S 2023, PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications. in ACM ICS 2023 - Proceedings of the International Conference on Supercomputing. Proceedings of the International Conference on Supercomputing, Association for Computing Machinery, pp. 167-179, 37th ACM International Conference on Supercomputing, ICS 2023, Orlando, United States, 06/21/23. https://doi.org/10.1145/3577193.3593705

PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications. / Zhang, Lingqi; Wahib, Mohamed; Chen, Peng et al.
ACM ICS 2023 - Proceedings of the International Conference on Supercomputing. Association for Computing Machinery, 2023. p. 167-179 (Proceedings of the International Conference on Supercomputing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - PERKS

T2 - 37th ACM International Conference on Supercomputing, ICS 2023

AU - Zhang, Lingqi

AU - Wahib, Mohamed

AU - Chen, Peng

AU - Meng, Jintao

AU - Wang, Xiao

AU - Endo, Toshio

AU - Matsuoka, Satoshi

PY - 2023/6/21

Y1 - 2023/6/21

N2 - Iterative memory-bound solvers commonly occur in HPC codes. Typical GPU implementations have a loop on the host side that invokes the GPU kernel as much as time/algorithm steps there are. The termination of each kernel implicitly acts the barrier required after advancing the solution every time step. We propose an execution model for running memory-bound iterative GPU kernels: PERsistent KernelS (PERKS). In this model, the time loop is moved inside persistent kernel, and device-wide barriers are used for synchronization. We then reduce the traffic to device memory by caching subset of the output in each time step in the unused registers and shared memory. PERKS can be generalized to any iterative solver: they largely independent of the solver's implementation. We explain the design principle of PERKS and demonstrate effectiveness of PERKS for a wide range of iterative 2D/3D stencil benchmarks (geomean speedup of 2.12x for 2D stencils and 1.24x for 3D stencils over state-of-art libraries), and a Krylov subspace conjugate gradient solver (geomean speedup of 4.86x in smaller SpMV datasets from SuiteSparse and 1.43x in larger SpMV datasets over a state-of-art library). All PERKS-based implementations available at: https://github.com/neozhang307/PERKS.

AB - Iterative memory-bound solvers commonly occur in HPC codes. Typical GPU implementations have a loop on the host side that invokes the GPU kernel as much as time/algorithm steps there are. The termination of each kernel implicitly acts the barrier required after advancing the solution every time step. We propose an execution model for running memory-bound iterative GPU kernels: PERsistent KernelS (PERKS). In this model, the time loop is moved inside persistent kernel, and device-wide barriers are used for synchronization. We then reduce the traffic to device memory by caching subset of the output in each time step in the unused registers and shared memory. PERKS can be generalized to any iterative solver: they largely independent of the solver's implementation. We explain the design principle of PERKS and demonstrate effectiveness of PERKS for a wide range of iterative 2D/3D stencil benchmarks (geomean speedup of 2.12x for 2D stencils and 1.24x for 3D stencils over state-of-art libraries), and a Krylov subspace conjugate gradient solver (geomean speedup of 4.86x in smaller SpMV datasets from SuiteSparse and 1.43x in larger SpMV datasets over a state-of-art library). All PERKS-based implementations available at: https://github.com/neozhang307/PERKS.

KW - GPU

KW - iterative solvers

KW - memory-bound

KW - persistent kernel

UR - https://www.scopus.com/pages/publications/85168409218

U2 - 10.1145/3577193.3593705

DO - 10.1145/3577193.3593705

M3 - Conference contribution

AN - SCOPUS:85168409218

T3 - Proceedings of the International Conference on Supercomputing

SP - 167

EP - 179

BT - ACM ICS 2023 - Proceedings of the International Conference on Supercomputing

PB - Association for Computing Machinery

Y2 - 21 June 2023 through 23 June 2023

ER -

Zhang L, Wahib M, Chen P, Meng J, Wang X, Endo T et al. PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications. In ACM ICS 2023 - Proceedings of the International Conference on Supercomputing. Association for Computing Machinery. 2023. p. 167-179. (Proceedings of the International Conference on Supercomputing). doi: 10.1145/3577193.3593705

PERKS: A Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this