Abstract
The heterogeneity of HPC systems requires efficient host-to-device porting of compute kernels and high-bandwidth data communication. This capability varies from one system to another depending on system architectures and environments. New vendors such as AMD and Intel are entering the GPU field, creating a software portability challenge. Major scientific simulation code bases rely on Fortran and require portable programming models for performance porting to HPC systems with high software productivity. Even though OpenMP target offloading features support portability, most Fortran-OpenMP code bases face significant challenges. Hence, in this work, we motivated an evaluation of a) the computing capability of heterogeneous systems for Fortran-OpenMP-based multi-physics code bases, and b) the performance portability of the astrophysical supernova simulation code Flash-X on heterogeneous systems. For this study, three HPC systems were chosen: Sunspot, a test-bed system of the Intel-PVC GPU featured supercomputer Aurora and Polaris, an NVIDIA system accelerated by A100 GPU, both located at the Argonne Leadership Computing Facility (ALCF), and the AMD-MI250-based Frontier at the Oak Ridge Leadership Computing Facility (OLCF). We discuss challenges and solutions for performance porting the compute-intensive module Thornado, which can be incorporated as an external library in Flash-X to model neutrino transport. We show that the performance of test apps improved by approximately 24× using the relevant optimization strategies + compiler-and-system updates. Further, this study helped improve the intel OneAPI-OpenMP compiler by providing bug reports and reproducers internally.
| Original language | English |
|---|---|
| Title of host publication | Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings |
| Editors | Alexis Espinosa, Maciej Cytowski, Michael Klemm, Bronis R. de Supinski, Jannis Klinkenberg |
| Publisher | Springer Science and Business Media Deutschland GmbH |
| Pages | 16-30 |
| Number of pages | 15 |
| ISBN (Print) | 9783031725661 |
| DOIs | |
| State | Published - 2024 |
| Event | 20th International Workshop on OpenMP, IWOMP 2024 - Perth, Australia Duration: Sep 23 2024 → Sep 25 2024 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 15195 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | 20th International Workshop on OpenMP, IWOMP 2024 |
|---|---|
| Country/Territory | Australia |
| City | Perth |
| Period | 09/23/24 → 09/25/24 |
Funding
This work was supported by the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357, and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of two U.S. Department of Energy organizations (Office of Science and the National Nuclear Security Administration). This research also used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. We also gratefully acknowledge the computing resources provided and operated by the Joint Laboratory for System Evaluation (JLSE) at Argonne National Laboratory(http://energy.gov/downloads/doe-public-access-plan).We extend our gratitude to Shaoping Quan, Dahai Guo, and William Dieter from Intel for their invaluable help and guidance in successfully completing this work. We would also like to thank Colleen Bertoni and Thomas Applencourt from Argonne for their fruitful discussions and timely guidance.
Keywords
- Concurrent multi-tasking
- Fortran-OpenMP code base
- Intel’s PVC GPU
- Multi-physics toolkit
- OpenACC
- OpenMP target offloading
- Porting to heterogeneous systems