CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Aaron Jarmusch; Felipe Cabarcas; Swaroop Pophale; Andrew Kallai; Johannes Doerfert; Luke Peyralans; Seyong Lee; Joel Denny; Sunita Chandrasekaran

doi:10.1007/978-3-031-72567-8_8

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Aaron Jarmusch
, Felipe Cabarcas
, Swaroop Pophale
, Andrew Kallai
, Johannes Doerfert
, Luke Peyralans
, Seyong Lee
, Joel Denny
, Sunita Chandrasekaran

Programming Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Original language	English
Title of host publication	Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings
Editors	Alexis Espinosa, Maciej Cytowski, Michael Klemm, Bronis R. de Supinski, Jannis Klinkenberg
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	111-125
Number of pages	15
ISBN (Print)	9783031725661
DOIs	https://doi.org/10.1007/978-3-031-72567-8_8
State	Published - 2024
Event	20th International Workshop on OpenMP, IWOMP 2024 - Perth, Australia Duration: Sep 23 2024 → Sep 25 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	15195 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th International Workshop on OpenMP, IWOMP 2024
Country/Territory	Australia
City	Perth
Period	09/23/24 → 09/25/24

Funding

This research used resources of the OLCF at ORNL supported by the Office of Science of the U.S. DOE under Contract No. DE-AC05-00OR22725; used resources of the ALCF, a U.S. DOE Office of Science user facility at Argonne National Laboratory and is based on research supported by the U.S. DOE Office of Science-ASCR program, under Contract No. DE-AC02-06CH11357; used resources of NERSC, a U.S. DOE Office of Science User Facility located at LBNL, operated under Contract No. DE-AC02-05CH11231 using NERSC ERCAP0029463. This material is also based upon work supported by the U.S. DOE under Contract DE-FOA-0003177, S4PST: Next Generation Science Software Technologies Project.

Keywords

Benchmarking
CI
Compiler
OpenMP
Validation

Access to Document

10.1007/978-3-031-72567-8_8

Cite this

Jarmusch, A., Cabarcas, F., Pophale, S., Kallai, A., Doerfert, J., Peyralans, L., Lee, S., Denny, J., & Chandrasekaran, S. (2024). CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations. In A. Espinosa, M. Cytowski, M. Klemm, B. R. de Supinski, & J. Klinkenberg (Eds.), Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings (pp. 111-125). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15195 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-72567-8_8

Jarmusch, Aaron ; Cabarcas, Felipe ; Pophale, Swaroop et al. / CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations. Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings. editor / Alexis Espinosa ; Maciej Cytowski ; Michael Klemm ; Bronis R. de Supinski ; Jannis Klinkenberg. Springer Science and Business Media Deutschland GmbH, 2024. pp. 111-125 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{1e4e8b5dac8c4b2bbe20f743e8157c2f,

title = "CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations",

abstract = "Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.",

keywords = "Benchmarking, CI, Compiler, OpenMP, Validation",

author = "Aaron Jarmusch and Felipe Cabarcas and Swaroop Pophale and Andrew Kallai and Johannes Doerfert and Luke Peyralans and Seyong Lee and Joel Denny and Sunita Chandrasekaran",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; 20th International Workshop on OpenMP, IWOMP 2024 ; Conference date: 23-09-2024 Through 25-09-2024",

year = "2024",

doi = "10.1007/978-3-031-72567-8\_8",

language = "English",

isbn = "9783031725661",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "111--125",

editor = "Alexis Espinosa and Maciej Cytowski and Michael Klemm and \{de Supinski\}, \{Bronis R.\} and Jannis Klinkenberg",

booktitle = "Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings",

}

Jarmusch, A, Cabarcas, F, Pophale, S, Kallai, A, Doerfert, J, Peyralans, L, Lee, S , Denny, J & Chandrasekaran, S 2024, CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations. in A Espinosa, M Cytowski, M Klemm, BR de Supinski & J Klinkenberg (eds), Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 15195 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 111-125, 20th International Workshop on OpenMP, IWOMP 2024, Perth, Australia, 09/23/24. https://doi.org/10.1007/978-3-031-72567-8_8

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations. / Jarmusch, Aaron; Cabarcas, Felipe; Pophale, Swaroop et al.
Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings. ed. / Alexis Espinosa; Maciej Cytowski; Michael Klemm; Bronis R. de Supinski; Jannis Klinkenberg. Springer Science and Business Media Deutschland GmbH, 2024. p. 111-125 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15195 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

AU - Jarmusch, Aaron

AU - Cabarcas, Felipe

AU - Pophale, Swaroop

AU - Kallai, Andrew

AU - Doerfert, Johannes

AU - Peyralans, Luke

AU - Lee, Seyong

AU - Denny, Joel

AU - Chandrasekaran, Sunita

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024

Y1 - 2024

N2 - Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

AB - Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

KW - Benchmarking

KW - CI

KW - Compiler

KW - OpenMP

KW - Validation

UR - https://www.scopus.com/pages/publications/85205343918

U2 - 10.1007/978-3-031-72567-8_8

DO - 10.1007/978-3-031-72567-8_8

M3 - Conference contribution

AN - SCOPUS:85205343918

SN - 9783031725661

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 111

EP - 125

BT - Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings

A2 - Espinosa, Alexis

A2 - Cytowski, Maciej

A2 - Klemm, Michael

A2 - de Supinski, Bronis R.

A2 - Klinkenberg, Jannis

PB - Springer Science and Business Media Deutschland GmbH

T2 - 20th International Workshop on OpenMP, IWOMP 2024

Y2 - 23 September 2024 through 25 September 2024

ER -

Jarmusch A, Cabarcas F, Pophale S, Kallai A, Doerfert J, Peyralans L et al. CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations. In Espinosa A, Cytowski M, Klemm M, de Supinski BR, Klinkenberg J, editors, Advancing OpenMP for Future Accelerators - 20th International Workshop on OpenMP, IWOMP 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 111-125. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-72567-8_8