Early experiences on the OLCF Frontier system with AthenaPK and Parthenon-Hydro

John K. Holmen; Philipp Grete; Verónica G. Melesse Vergara

doi:10.1002/cpe.8069

Early experiences on the OLCF Frontier system with AthenaPK and Parthenon-Hydro

John K. Holmen, Philipp Grete, Verónica G. Melesse Vergara

National Center for Computational Scienc

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

The Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, “real-world” user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per-cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini-application stressing the core functionality of a performance portable block-structured adaptive mesh refinement framework, Parthenon-Hydro. These results show good scaling characteristics to the full system. At the largest scale, the Parthenon-Hydro miniapp reaches a total of (Formula presented.) zone-cycles/s on 9216 nodes (73,728 logical GPUs) at (Formula presented.) 92% weak scaling parallel efficiency (starting from a single node using a second-order, finite-volume method).

Original language	English
Article number	e8069
Journal	Concurrency and Computation: Practice and Experience
Volume	36
Issue number	13
DOIs	https://doi.org/10.1002/cpe.8069
State	Published - Jun 10 2024

Funding

The authors would like to thank the Parthenon and Kokkos communities for being open, approachable and supportive. The authors would also like the thank the OLCF for early access to Frontier. This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE‐AC05‐00OR22725. This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska‐Curie Grant agreement No. 101030214.

Funders	Funder number
Office of Science	DE‐AC05‐00OR22725
Horizon 2020	101030214

Keywords

adaptive mesh refinement
high-performance computing
parallel computing
performance portability

Access to Document

10.1002/cpe.8069

Cite this

@article{c3b08c50b53b40d7b239ee720d640dc8,

title = "Early experiences on the OLCF Frontier system with AthenaPK and Parthenon-Hydro",

abstract = "The Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, “real-world” user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per-cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini-application stressing the core functionality of a performance portable block-structured adaptive mesh refinement framework, Parthenon-Hydro. These results show good scaling characteristics to the full system. At the largest scale, the Parthenon-Hydro miniapp reaches a total of (Formula presented.) zone-cycles/s on 9216 nodes (73,728 logical GPUs) at (Formula presented.) 92% weak scaling parallel efficiency (starting from a single node using a second-order, finite-volume method).",

keywords = "adaptive mesh refinement, high-performance computing, parallel computing, performance portability",

author = "Holmen, {John K.} and Philipp Grete and {Melesse Vergara}, {Ver{\'o}nica G.}",

note = "Publisher Copyright: {\textcopyright} 2024 John Wiley & Sons Ltd.",

year = "2024",

month = jun,

day = "10",

doi = "10.1002/cpe.8069",

language = "English",

volume = "36",

journal = "Concurrency and Computation: Practice and Experience",

issn = "1532-0626",

publisher = "wiley",

number = "13",

}

TY - JOUR

T1 - Early experiences on the OLCF Frontier system with AthenaPK and Parthenon-Hydro

AU - Holmen, John K.

AU - Grete, Philipp

AU - Melesse Vergara, Verónica G.

PY - 2024/6/10

Y1 - 2024/6/10

N2 - The Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, “real-world” user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per-cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini-application stressing the core functionality of a performance portable block-structured adaptive mesh refinement framework, Parthenon-Hydro. These results show good scaling characteristics to the full system. At the largest scale, the Parthenon-Hydro miniapp reaches a total of (Formula presented.) zone-cycles/s on 9216 nodes (73,728 logical GPUs) at (Formula presented.) 92% weak scaling parallel efficiency (starting from a single node using a second-order, finite-volume method).

AB - The Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, “real-world” user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per-cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini-application stressing the core functionality of a performance portable block-structured adaptive mesh refinement framework, Parthenon-Hydro. These results show good scaling characteristics to the full system. At the largest scale, the Parthenon-Hydro miniapp reaches a total of (Formula presented.) zone-cycles/s on 9216 nodes (73,728 logical GPUs) at (Formula presented.) 92% weak scaling parallel efficiency (starting from a single node using a second-order, finite-volume method).

KW - adaptive mesh refinement

KW - high-performance computing

KW - parallel computing

KW - performance portability

UR - http://www.scopus.com/inward/record.url?scp=85186926608&partnerID=8YFLogxK

U2 - 10.1002/cpe.8069

DO - 10.1002/cpe.8069

M3 - Article

AN - SCOPUS:85186926608

SN - 1532-0626

VL - 36

JO - Concurrency and Computation: Practice and Experience

JF - Concurrency and Computation: Practice and Experience

IS - 13

M1 - e8069

ER -

Early experiences on the OLCF Frontier system with AthenaPK and Parthenon-Hydro

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this