KokkACC: Enhancing Kokkos with OpenACC

Pedro Valero-Lara; Seyong Lee; Marc Gonzalez-Tallada; Joel Denny; Jeffrey S. Vetter

doi:10.1109/WACCPD56842.2022.00009

KokkACC: Enhancing Kokkos with OpenACC

Pedro Valero-Lara, Seyong Lee, Marc Gonzalez-Tallada, Joel Denny, Jeffrey S. Vetter

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

Template metaprogramming is gaining popularity as a high-level solution for achieving performance portability on heterogeneous computing resources. Kokkos is a representative approach that offers programmers high-level abstractions for generic programming while most of the device-specific code generation and optimizations are delegated to the compiler through template specializations. For this, Kokkos provides a set of device-specific code specializations in multiple back ends, such as CUDA and HIP. Unlike CUDA or HIP, OpenACC is a high-level and directive-based programming model. This descriptive model allows developers to insert hints (pragmas) into their code that help the compiler to parallelize the code. The compiler is responsible for the transformation of the code, which is completely transparent to the programmer. This paper presents an OpenACC back end for Kokkos: KokkACC. As an alternative to Kokkos's existing device-specific back ends, KokkACC is a multi-architecture back end providing a high-productivity programming environment enabled by OpenACC's high-level and descriptive programming model. Moreover, we have observed competitive performance; in some cases, KokkACC is faster (up to 9×) than NVIDIA's CUDA back end and much faster than OpenMP's GPU offloading back end. This work also includes implementation details and a detailed performance study conducted with a set of mini-benchmarks (AXPY and DOT product) and three mini-apps (LULESH, miniFE and SNAP, a LAMMPS proxy mini-app).

Original language	English
Title of host publication	Proceedings of WACCPD 2022
Subtitle of host publication	9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	32-42
Number of pages	11
ISBN (Electronic)	9781665490191
DOIs	https://doi.org/10.1109/WACCPD56842.2022.00009
State	Published - 2022
Event	9th Workshop on Accelerator Programming Using Directives, WACCPD 2022 - Dallas, United States Duration: Nov 13 2022 → Nov 18 2022

Publication series

Name	Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	9th Workshop on Accelerator Programming Using Directives, WACCPD 2022
Country/Territory	United States
City	Dallas
Period	11/13/22 → 11/18/22

Funding

This research used resources of the Experimental Computing Laboratory and the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which are supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This research was supported in part by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a non-exclusive, paid up, irrevocable, world-wide license to publish or reproduce the published form of the manuscript, or allow others to do so, for U.S. Government purposes. The DOE will provide public access to these results in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-accessplan). This research used resources of the Experimental Computing Laboratory and the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which are supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This research was supported in part by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a non-exclusive, paid up, irrevocable, world-wide license to publish or reproduce the published form of the manuscript, or allow others to do so, for U.S. Government purposes. The DOE will provide public access to these results in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

C++ Metaprogramming
CUDA
Kokkos
OpenACC
OpenMP Target
Parallel Programming Models

Access to Document

10.1109/WACCPD56842.2022.00009

Cite this

Valero-Lara, P., Lee, S., Gonzalez-Tallada, M., Denny, J., & Vetter, J. S. (2022). KokkACC: Enhancing Kokkos with OpenACC. In Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 32-42). (Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACCPD56842.2022.00009

Valero-Lara, Pedro ; Lee, Seyong ; Gonzalez-Tallada, Marc et al. / KokkACC : Enhancing Kokkos with OpenACC. Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 32-42 (Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{520810af637d48c6a81be3d7b40e7339,

title = "KokkACC: Enhancing Kokkos with OpenACC",

abstract = "Template metaprogramming is gaining popularity as a high-level solution for achieving performance portability on heterogeneous computing resources. Kokkos is a representative approach that offers programmers high-level abstractions for generic programming while most of the device-specific code generation and optimizations are delegated to the compiler through template specializations. For this, Kokkos provides a set of device-specific code specializations in multiple back ends, such as CUDA and HIP. Unlike CUDA or HIP, OpenACC is a high-level and directive-based programming model. This descriptive model allows developers to insert hints (pragmas) into their code that help the compiler to parallelize the code. The compiler is responsible for the transformation of the code, which is completely transparent to the programmer. This paper presents an OpenACC back end for Kokkos: KokkACC. As an alternative to Kokkos's existing device-specific back ends, KokkACC is a multi-architecture back end providing a high-productivity programming environment enabled by OpenACC's high-level and descriptive programming model. Moreover, we have observed competitive performance; in some cases, KokkACC is faster (up to 9×) than NVIDIA's CUDA back end and much faster than OpenMP's GPU offloading back end. This work also includes implementation details and a detailed performance study conducted with a set of mini-benchmarks (AXPY and DOT product) and three mini-apps (LULESH, miniFE and SNAP, a LAMMPS proxy mini-app).",

keywords = "C++ Metaprogramming, CUDA, Kokkos, OpenACC, OpenMP Target, Parallel Programming Models",

author = "Pedro Valero-Lara and Seyong Lee and Marc Gonzalez-Tallada and Joel Denny and Vetter, {Jeffrey S.}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 9th Workshop on Accelerator Programming Using Directives, WACCPD 2022 ; Conference date: 13-11-2022 Through 18-11-2022",

year = "2022",

doi = "10.1109/WACCPD56842.2022.00009",

language = "English",

series = "Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "32--42",

booktitle = "Proceedings of WACCPD 2022",

}

Valero-Lara, P , Lee, S, Gonzalez-Tallada, M, Denny, J & Vetter, JS 2022, KokkACC: Enhancing Kokkos with OpenACC. in Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 32-42, 9th Workshop on Accelerator Programming Using Directives, WACCPD 2022, Dallas, United States, 11/13/22. https://doi.org/10.1109/WACCPD56842.2022.00009

KokkACC: Enhancing Kokkos with OpenACC. / Valero-Lara, Pedro ; Lee, Seyong; Gonzalez-Tallada, Marc et al.
Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2022. p. 32-42 (Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - KokkACC

T2 - 9th Workshop on Accelerator Programming Using Directives, WACCPD 2022

AU - Valero-Lara, Pedro

AU - Lee, Seyong

AU - Gonzalez-Tallada, Marc

AU - Denny, Joel

AU - Vetter, Jeffrey S.

PY - 2022

Y1 - 2022

N2 - Template metaprogramming is gaining popularity as a high-level solution for achieving performance portability on heterogeneous computing resources. Kokkos is a representative approach that offers programmers high-level abstractions for generic programming while most of the device-specific code generation and optimizations are delegated to the compiler through template specializations. For this, Kokkos provides a set of device-specific code specializations in multiple back ends, such as CUDA and HIP. Unlike CUDA or HIP, OpenACC is a high-level and directive-based programming model. This descriptive model allows developers to insert hints (pragmas) into their code that help the compiler to parallelize the code. The compiler is responsible for the transformation of the code, which is completely transparent to the programmer. This paper presents an OpenACC back end for Kokkos: KokkACC. As an alternative to Kokkos's existing device-specific back ends, KokkACC is a multi-architecture back end providing a high-productivity programming environment enabled by OpenACC's high-level and descriptive programming model. Moreover, we have observed competitive performance; in some cases, KokkACC is faster (up to 9×) than NVIDIA's CUDA back end and much faster than OpenMP's GPU offloading back end. This work also includes implementation details and a detailed performance study conducted with a set of mini-benchmarks (AXPY and DOT product) and three mini-apps (LULESH, miniFE and SNAP, a LAMMPS proxy mini-app).

AB - Template metaprogramming is gaining popularity as a high-level solution for achieving performance portability on heterogeneous computing resources. Kokkos is a representative approach that offers programmers high-level abstractions for generic programming while most of the device-specific code generation and optimizations are delegated to the compiler through template specializations. For this, Kokkos provides a set of device-specific code specializations in multiple back ends, such as CUDA and HIP. Unlike CUDA or HIP, OpenACC is a high-level and directive-based programming model. This descriptive model allows developers to insert hints (pragmas) into their code that help the compiler to parallelize the code. The compiler is responsible for the transformation of the code, which is completely transparent to the programmer. This paper presents an OpenACC back end for Kokkos: KokkACC. As an alternative to Kokkos's existing device-specific back ends, KokkACC is a multi-architecture back end providing a high-productivity programming environment enabled by OpenACC's high-level and descriptive programming model. Moreover, we have observed competitive performance; in some cases, KokkACC is faster (up to 9×) than NVIDIA's CUDA back end and much faster than OpenMP's GPU offloading back end. This work also includes implementation details and a detailed performance study conducted with a set of mini-benchmarks (AXPY and DOT product) and three mini-apps (LULESH, miniFE and SNAP, a LAMMPS proxy mini-app).

KW - C++ Metaprogramming

KW - CUDA

KW - Kokkos

KW - OpenACC

KW - OpenMP Target

KW - Parallel Programming Models

UR - http://www.scopus.com/inward/record.url?scp=85147965415&partnerID=8YFLogxK

U2 - 10.1109/WACCPD56842.2022.00009

DO - 10.1109/WACCPD56842.2022.00009

M3 - Conference contribution

AN - SCOPUS:85147965415

T3 - Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 32

EP - 42

BT - Proceedings of WACCPD 2022

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 13 November 2022 through 18 November 2022

ER -

Valero-Lara P , Lee S, Gonzalez-Tallada M, Denny J , Vetter JS. KokkACC: Enhancing Kokkos with OpenACC. In Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2022. p. 32-42. (Proceedings of WACCPD 2022: 9th Workshop on Accelerator Programming Using Directives, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/WACCPD56842.2022.00009

KokkACC: Enhancing Kokkos with OpenACC

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this