First Experiences in Performance Benchmarking with the New SPEChpc 2021 Suites

Holger Brunst, Sunita Chandrasekaran, Florina M. Ciorba, Nick Hagerty, Robert Henschel, Guido Juckeland, Junjie Li, Veronica G.Melesse Vergara, Sandra Wienke, Miguel Zavala

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

13 Scopus citations

Abstract

Modern High Performance Computing (HPC) sys-tems are built with innovative system architectures and novel programming models to further push the speed limit of computing. The increased complexity poses challenges for performance portability and performance evaluation. The Standard Perfor-mance Evaluation Corporation (SPEC) has a long history of producing industry-standard benchmarks for modern computer systems. SPEC's newly released SPEChpc 2021 benchmark suites, developed by the High Performance Group, are a bold attempt to provide a fair and objective benchmarking tool designed for state-of-the-art HPC systems. With the support of multiple host and accelerator programming models, the suites are portable across both homogeneous and heterogeneous architectures. Different workloads are developed to fit system sizes ranging from a few compute nodes to a few hundred compute nodes. In this work we present our first experiences in performance benchmarking the new SPEChpc2021 suites and evaluate their portability and basic performance characteristics on various popular and emerging HPC architectures, including x86 CPU, NVIDIA GPU, and AMD GPU. This study provides a first-hand experience of executing the SPEChpc 2021 suites at scale on production HPC systems, discusses real-world use cases, and serves as an initial guideline for using the benchmark suites.

Original languageEnglish
Title of host publicationProceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022
EditorsMaria Fazio, Dhabaleswar K. Panda, Radu Prodan, Valeria Cardellini, Burak Kantarci, Omer Rana, Massimo Villari
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages675-684
Number of pages10
ISBN (Electronic)9781665499569
DOIs
StatePublished - 2022
Event22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022 - Taormina, Italy
Duration: May 16 2022May 19 2022

Publication series

NameProceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022

Conference

Conference22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022
Country/TerritoryItaly
CityTaormina
Period05/16/2205/19/22

Funding

The authors would like to acknowledge a number of facilities and grants for their support, but first and foremost, we would like to thank the full SPEC HPG group for the tremendous effort behind developing and releasing the SPEChpc 2021 benchmark suites. Then, the Center for Information Services and HPC at TU Dresden for providing its facilities for high throughput calculations; RWTH Aachen University under project rwth0663 for supporting simulations on their computing resources; supported by NSF under grant no. 1814609; Gauss Centre for Supercomputing e.V. for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUWELS at Jülich Supercomputing Centre (JSC); Oak Ridge Leadership Computing Facility, a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725 for resources used; the Frontera supercomputer at TACC funded by NSF for large scaling calculations and profiling; the Swiss PASC initiative via the SPH-EXA project; the Swiss State Secretariat for Education, Research and Innovation (SERI).

Keywords

  • HPC
  • HPG
  • MPI
  • MPI+X
  • OpenACC
  • OpenMP
  • SPEC
  • SPEChpc 2021
  • benchmarks
  • heterogeneity
  • offloading
  • performance benchmarking and analysis

Fingerprint

Dive into the research topics of 'First Experiences in Performance Benchmarking with the New SPEChpc 2021 Suites'. Together they form a unique fingerprint.

Cite this