A MultiGPU Performance-Portable Solution for Array Programming Based on Kokkos

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

MultiGPU nodes are widely used in high-performance computing and data centers. However, current programming models do not provide transparent, and portable support for automatically targeting multiple GPUs within a node. In this paper, we describe a new application programming interface based on the Kokkos programming model to enable array computation on multiple GPUs in a transparent and portable way across both NVIDIA and AMD GPUs. We implement different variations of this technique to accommodate the exchange of stencils, and we provide autotuning to select the proper number of GPUs, depending on the computational cost of the operations to be computed on arrays. We evaluate our multiGPU extension on Summit (#5 TOP500), with six NVIDIA V100 Volta GPUs per node, and Crusher that contains identical hardware/software as Frontier (#1 TOP500), with four AMD MI250X GPUs, each with 2 Graphics Compute Dies (GCDs) for a total of 8 GCDs per node. We also compare the performance of this solution against the use of MPI + Kokkos. Our evaluation shows that the new Kokkos solution provides good scalability for many GPUs. than MPI + Kokkos.

Original languageEnglish
Title of host publicationARRAY 2023 - Proceedings of the 9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming, Co-located with PLDI 2023
EditorsTroels Henriksen, Artjoms Sinkarovs
PublisherAssociation for Computing Machinery, Inc
Pages1-12
Number of pages12
ISBN (Electronic)9798400701696
DOIs
StatePublished - Jun 6 2023
Event9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming, ARRAY 2023, co-located with PLDI 2023 - Orlando, United States
Duration: Jun 18 2023Jun 18 2023

Publication series

NameARRAY 2023 - Proceedings of the 9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming, Co-located with PLDI 2023

Conference

Conference9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming, ARRAY 2023, co-located with PLDI 2023
Country/TerritoryUnited States
CityOrlando
Period06/18/2306/18/23

Funding

This research used resources of the Oak Ridge Leadership Computing Facility and the Experimental Computing Laboratory at the Oak Ridge National Laboratory, which is supported by DOE’s Office of Science under Contract No. DE-AC05-00OR22725. This research was supported in part by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the DOE’s Office of Science and the National Nuclear Security Administration. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the DOE. The publisher, by accepting the article for publication, acknowledges that the US Government retains a non-exclusive, paid up, irrevocable, world-wide license to publish or reproduce the published form of the manuscript, or allow others to do so, for US Government purposes. The DOE will provide public access to these results in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). This research used resources of the Oak Ridge Leadership Computing Facility and the Experimental Computing Laboratory at the Oak Ridge National Laboratory, which is supported by DOE's Office of Science under Contract No. DEAC05- 00OR22725. This research was supported in part by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the DOE's Office of Science and the National Nuclear Security Administration. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05- 00OR22725 with the DOE. The publisher, by accepting the article for publication, acknowledges that the US Government retains a non-exclusive, paid up, irrevocable, worldwide license to publish or reproduce the published form of the manuscript, or allow others to do so, for US Government purposes. The DOE will provide public access to these results in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

  • Array programming
  • C++ metaprogramming
  • CUDA
  • HIP
  • Kokkos
  • MPI
  • autotuning
  • multiGPU
  • parallel programming model
  • performance portability
  • productivity

Fingerprint

Dive into the research topics of 'A MultiGPU Performance-Portable Solution for Array Programming Based on Kokkos'. Together they form a unique fingerprint.

Cite this