Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes

William F. Godoy; Pedro Valero-Lara; T. Elise Dettling; Christian Trefftz; Ian Jorquera; Thomas Sheehy; Ross G. Miller; Marc Gonzalez-Tallada; Jeffrey S. Vetter; Valentin Churavy

doi:10.1109/IPDPSW59300.2023.00068

Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes

William F. Godoy, Pedro Valero-Lara, T. Elise Dettling, Christian Trefftz, Ian Jorquera, Thomas Sheehy, Ross G. Miller, Marc Gonzalez-Tallada, Jeffrey S. Vetter, Valentin Churavy

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

We explore the performance and portability of the high-level programming models: the LLVM-based Julia and Python/Numba, and Kokkos on high-performance computing (HPC) nodes: AMD Epyc CPUs and MI250X graphical processing units (GPUs) on Frontier's test bed Crusher system and Ampere's Arm-based CPUs and NVIDIA's A100 GPUs on the Wombat system at the Oak Ridge Leadership Computing Facilities. We compare the default performance of a hand-rolled dense matrix multiplication algorithm on CPUs against vendor-compiled C/OpenMP implementations, and on each GPU against CUDA and HIP. Rather than focusing on the kernel optimization per-se, we select this naive approach to resemble exploratory work in science and as a lower-bound for performance to isolate the effect of each programming model. Julia and Kokkos perform comparably with C/OpenMP on CPUs, while Julia implementations are competitive with CUDA and HIP on GPUs. Performance gaps are identified on NVIDIA A100 GPUs for Julia's single precision and Kokkos, and for Python/Numba in all scenarios. We also comment on half-precision support, productivity, performance portability metrics, and platform readiness. We expect to contribute to the understanding and direction for high-level, high-productivity languages in HPC as the first-generation exascale systems are deployed.

Original language	English
Title of host publication	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	373-382
Number of pages	10
ISBN (Electronic)	9798350311990
DOIs	https://doi.org/10.1109/IPDPSW59300.2023.00068
State	Published - 2023
Event	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 - St. Petersburg, United States Duration: May 15 2023 → May 19 2023

Publication series

Name	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

Conference

Conference	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023
Country/Territory	United States
City	St. Petersburg
Period	05/15/23 → 05/19/23

Funding

This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaboratvi e effort of the US Department of Energy Office of Science and the National Nuclear Security Administration. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the US Department of Energy under Contract No. DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The publisher acknowledges the US government license to provide public access under the DOE Public Access Plan (https://energy.gov/downloads/ doe-public-access-plan).

Funders	Funder number
DOE Public Access Plan
U.S. Department of Energy	DE-AC05-00OR22725
Office of Science
National Nuclear Security Administration

Keywords

Exascale
GPU
HPC
Julia
Kokkos
LLVM
OpenMP
Performance
Portability
Python/Numba

Access to Document

10.1109/IPDPSW59300.2023.00068

Cite this

Godoy, W. F., Valero-Lara, P., Dettling, T. E., Trefftz, C., Jorquera, I., Sheehy, T., Miller, R. G., Gonzalez-Tallada, M., Vetter, J. S., & Churavy, V. (2023). Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes. In 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 (pp. 373-382). (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPSW59300.2023.00068

Godoy, William F. ; Valero-Lara, Pedro ; Dettling, T. Elise et al. / Evaluating performance and portability of high-level programming models : Julia, Python/Numba, and Kokkos on exascale nodes. 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 373-382 (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023).

@inproceedings{e86aedba39744bbf81c434c5562f33ec,

title = "Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes",

abstract = "We explore the performance and portability of the high-level programming models: the LLVM-based Julia and Python/Numba, and Kokkos on high-performance computing (HPC) nodes: AMD Epyc CPUs and MI250X graphical processing units (GPUs) on Frontier's test bed Crusher system and Ampere's Arm-based CPUs and NVIDIA's A100 GPUs on the Wombat system at the Oak Ridge Leadership Computing Facilities. We compare the default performance of a hand-rolled dense matrix multiplication algorithm on CPUs against vendor-compiled C/OpenMP implementations, and on each GPU against CUDA and HIP. Rather than focusing on the kernel optimization per-se, we select this naive approach to resemble exploratory work in science and as a lower-bound for performance to isolate the effect of each programming model. Julia and Kokkos perform comparably with C/OpenMP on CPUs, while Julia implementations are competitive with CUDA and HIP on GPUs. Performance gaps are identified on NVIDIA A100 GPUs for Julia's single precision and Kokkos, and for Python/Numba in all scenarios. We also comment on half-precision support, productivity, performance portability metrics, and platform readiness. We expect to contribute to the understanding and direction for high-level, high-productivity languages in HPC as the first-generation exascale systems are deployed.",

keywords = "Exascale, GPU, HPC, Julia, Kokkos, LLVM, OpenMP, Performance, Portability, Python/Numba",

author = "Godoy, {William F.} and Pedro Valero-Lara and Dettling, {T. Elise} and Christian Trefftz and Ian Jorquera and Thomas Sheehy and Miller, {Ross G.} and Marc Gonzalez-Tallada and Vetter, {Jeffrey S.} and Valentin Churavy",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 ; Conference date: 15-05-2023 Through 19-05-2023",

year = "2023",

doi = "10.1109/IPDPSW59300.2023.00068",

language = "English",

series = "2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "373--382",

booktitle = "2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023",

}

Godoy, WF , Valero-Lara, P, Dettling, TE, Trefftz, C, Jorquera, I, Sheehy, T, Miller, RG, Gonzalez-Tallada, M, Vetter, JS & Churavy, V 2023, Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes. in 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023, Institute of Electrical and Electronics Engineers Inc., pp. 373-382, 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023, St. Petersburg, United States, 05/15/23. https://doi.org/10.1109/IPDPSW59300.2023.00068

Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes. / Godoy, William F.; Valero-Lara, Pedro; Dettling, T. Elise et al.
2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 373-382 (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Evaluating performance and portability of high-level programming models

T2 - 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

AU - Godoy, William F.

AU - Valero-Lara, Pedro

AU - Dettling, T. Elise

AU - Trefftz, Christian

AU - Jorquera, Ian

AU - Sheehy, Thomas

AU - Miller, Ross G.

AU - Gonzalez-Tallada, Marc

AU - Vetter, Jeffrey S.

AU - Churavy, Valentin

PY - 2023

Y1 - 2023

N2 - We explore the performance and portability of the high-level programming models: the LLVM-based Julia and Python/Numba, and Kokkos on high-performance computing (HPC) nodes: AMD Epyc CPUs and MI250X graphical processing units (GPUs) on Frontier's test bed Crusher system and Ampere's Arm-based CPUs and NVIDIA's A100 GPUs on the Wombat system at the Oak Ridge Leadership Computing Facilities. We compare the default performance of a hand-rolled dense matrix multiplication algorithm on CPUs against vendor-compiled C/OpenMP implementations, and on each GPU against CUDA and HIP. Rather than focusing on the kernel optimization per-se, we select this naive approach to resemble exploratory work in science and as a lower-bound for performance to isolate the effect of each programming model. Julia and Kokkos perform comparably with C/OpenMP on CPUs, while Julia implementations are competitive with CUDA and HIP on GPUs. Performance gaps are identified on NVIDIA A100 GPUs for Julia's single precision and Kokkos, and for Python/Numba in all scenarios. We also comment on half-precision support, productivity, performance portability metrics, and platform readiness. We expect to contribute to the understanding and direction for high-level, high-productivity languages in HPC as the first-generation exascale systems are deployed.

AB - We explore the performance and portability of the high-level programming models: the LLVM-based Julia and Python/Numba, and Kokkos on high-performance computing (HPC) nodes: AMD Epyc CPUs and MI250X graphical processing units (GPUs) on Frontier's test bed Crusher system and Ampere's Arm-based CPUs and NVIDIA's A100 GPUs on the Wombat system at the Oak Ridge Leadership Computing Facilities. We compare the default performance of a hand-rolled dense matrix multiplication algorithm on CPUs against vendor-compiled C/OpenMP implementations, and on each GPU against CUDA and HIP. Rather than focusing on the kernel optimization per-se, we select this naive approach to resemble exploratory work in science and as a lower-bound for performance to isolate the effect of each programming model. Julia and Kokkos perform comparably with C/OpenMP on CPUs, while Julia implementations are competitive with CUDA and HIP on GPUs. Performance gaps are identified on NVIDIA A100 GPUs for Julia's single precision and Kokkos, and for Python/Numba in all scenarios. We also comment on half-precision support, productivity, performance portability metrics, and platform readiness. We expect to contribute to the understanding and direction for high-level, high-productivity languages in HPC as the first-generation exascale systems are deployed.

KW - Exascale

KW - GPU

KW - HPC

KW - Julia

KW - Kokkos

KW - LLVM

KW - OpenMP

KW - Performance

KW - Portability

KW - Python/Numba

UR - http://www.scopus.com/inward/record.url?scp=85169299348&partnerID=8YFLogxK

U2 - 10.1109/IPDPSW59300.2023.00068

DO - 10.1109/IPDPSW59300.2023.00068

M3 - Conference contribution

AN - SCOPUS:85169299348

T3 - 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

SP - 373

EP - 382

BT - 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 15 May 2023 through 19 May 2023

ER -

Godoy WF , Valero-Lara P, Dettling TE, Trefftz C, Jorquera I, Sheehy T et al. Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes. In 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 373-382. (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023). doi: 10.1109/IPDPSW59300.2023.00068

Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this