HPC Benchmarking: Scaling Right and Looking Beyond the Average

Milan Radulovic; Kazi Asifuzzaman; Paul Carpenter; Petar Radojković; Eduard Ayguadé

doi:10.1007/978-3-319-96983-1_10

HPC Benchmarking: Scaling Right and Looking Beyond the Average

Milan Radulovic, Kazi Asifuzzaman, Paul Carpenter, Petar Radojković, Eduard Ayguadé

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads that quantifies the main performance bottlenecks. In this paper, we execute seven production HPC applications on a production HPC platform, and analyse the key performance bottlenecks: FLOPS performance and memory bandwidth congestion, and the implications on scaling out. We show that the results depend significantly on the number of execution processes and granularity of measurements. We therefore advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose that the FLOPS performance and memory bandwidth should be represented in terms of the proportions of time with low, moderate and severe utilization. We show that this gives much more precise and actionable evidence than the average.

Original language	English
Title of host publication	Euro-Par 2018
Subtitle of host publication	Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings
Editors	Massimo Torquati, Marco Aldinucci, Luca Padovani
Publisher	Springer Verlag
Pages	135-146
Number of pages	12
ISBN (Print)	9783319969824
DOIs	https://doi.org/10.1007/978-3-319-96983-1_10
State	Published - 2018
Externally published	Yes
Event	24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018 - Turin, Italy Duration: Aug 27 2018 → Aug 31 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11014 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018
Country/Territory	Italy
City	Turin
Period	08/27/18 → 08/31/18

Funding

This work was supported by the Spanish Ministry of Science and Technology (project TIN2015-65316-P), Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), Severo Ochoa Programme (SEV-2015-0493) of the Spanish Government; and the European Union’s Horizon 2020 research and innovation programme under ExaNoDe project (grant agreement No 671578). Acknowledgements. This work was supported by the Spanish Ministry of Science and Technology (project TIN2015-65316-P), Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), Severo Ochoa Programme (SEV-2015-0493) of the Spanish Government; and the European Union’s Horizon 2020 research and innovation programme under ExaNoDe project (grant agreement No 671578).

Funders	Funder number
Ministry of Science and Technology
Horizon 2020 Framework Programme	671578
Fundación Carmen y Severo Ochoa	SEV-2015-0493
Generalitat de Catalunya	2014-SGR-1272, 2014-SGR-1051
Ministry of Science and Technology, Israel
Ministerio de Ciencia y Tecnología	TIN2015-65316-P

Keywords

Bottlenecks
FLOPS
HPC applications
Memory bandwidth
Scaling-out

Access to Document

10.1007/978-3-319-96983-1_10

Cite this

Radulovic, M., Asifuzzaman, K., Carpenter, P., Radojković, P., & Ayguadé, E. (2018). HPC Benchmarking: Scaling Right and Looking Beyond the Average. In M. Torquati, M. Aldinucci, & L. Padovani (Eds.), Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings (pp. 135-146). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11014 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-96983-1_10

Radulovic, Milan ; Asifuzzaman, Kazi ; Carpenter, Paul et al. / HPC Benchmarking : Scaling Right and Looking Beyond the Average. Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings. editor / Massimo Torquati ; Marco Aldinucci ; Luca Padovani. Springer Verlag, 2018. pp. 135-146 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2291769d067045cea8e875fd45d5e837,

title = "HPC Benchmarking: Scaling Right and Looking Beyond the Average",

abstract = "Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads that quantifies the main performance bottlenecks. In this paper, we execute seven production HPC applications on a production HPC platform, and analyse the key performance bottlenecks: FLOPS performance and memory bandwidth congestion, and the implications on scaling out. We show that the results depend significantly on the number of execution processes and granularity of measurements. We therefore advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose that the FLOPS performance and memory bandwidth should be represented in terms of the proportions of time with low, moderate and severe utilization. We show that this gives much more precise and actionable evidence than the average.",

keywords = "Bottlenecks, FLOPS, HPC applications, Memory bandwidth, Scaling-out",

author = "Milan Radulovic and Kazi Asifuzzaman and Paul Carpenter and Petar Radojkovi{\'c} and Eduard Ayguad{\'e}",

note = "Publisher Copyright: {\textcopyright} 2018, Springer International Publishing AG, part of Springer Nature.; 24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018 ; Conference date: 27-08-2018 Through 31-08-2018",

year = "2018",

doi = "10.1007/978-3-319-96983-1_10",

language = "English",

isbn = "9783319969824",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "135--146",

editor = "Massimo Torquati and Marco Aldinucci and Luca Padovani",

booktitle = "Euro-Par 2018",

}

Radulovic, M, Asifuzzaman, K, Carpenter, P, Radojković, P & Ayguadé, E 2018, HPC Benchmarking: Scaling Right and Looking Beyond the Average. in M Torquati, M Aldinucci & L Padovani (eds), Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11014 LNCS, Springer Verlag, pp. 135-146, 24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018, Turin, Italy, 08/27/18. https://doi.org/10.1007/978-3-319-96983-1_10

HPC Benchmarking: Scaling Right and Looking Beyond the Average. / Radulovic, Milan; Asifuzzaman, Kazi; Carpenter, Paul et al.
Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings. ed. / Massimo Torquati; Marco Aldinucci; Luca Padovani. Springer Verlag, 2018. p. 135-146 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11014 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - HPC Benchmarking

T2 - 24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018

AU - Radulovic, Milan

AU - Asifuzzaman, Kazi

AU - Carpenter, Paul

AU - Radojković, Petar

AU - Ayguadé, Eduard

PY - 2018

Y1 - 2018

N2 - Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads that quantifies the main performance bottlenecks. In this paper, we execute seven production HPC applications on a production HPC platform, and analyse the key performance bottlenecks: FLOPS performance and memory bandwidth congestion, and the implications on scaling out. We show that the results depend significantly on the number of execution processes and granularity of measurements. We therefore advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose that the FLOPS performance and memory bandwidth should be represented in terms of the proportions of time with low, moderate and severe utilization. We show that this gives much more precise and actionable evidence than the average.

AB - Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads that quantifies the main performance bottlenecks. In this paper, we execute seven production HPC applications on a production HPC platform, and analyse the key performance bottlenecks: FLOPS performance and memory bandwidth congestion, and the implications on scaling out. We show that the results depend significantly on the number of execution processes and granularity of measurements. We therefore advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose that the FLOPS performance and memory bandwidth should be represented in terms of the proportions of time with low, moderate and severe utilization. We show that this gives much more precise and actionable evidence than the average.

KW - Bottlenecks

KW - FLOPS

KW - HPC applications

KW - Memory bandwidth

KW - Scaling-out

UR - http://www.scopus.com/inward/record.url?scp=85052939963&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-96983-1_10

DO - 10.1007/978-3-319-96983-1_10

M3 - Conference contribution

AN - SCOPUS:85052939963

SN - 9783319969824

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 135

EP - 146

BT - Euro-Par 2018

A2 - Torquati, Massimo

A2 - Aldinucci, Marco

A2 - Padovani, Luca

PB - Springer Verlag

Y2 - 27 August 2018 through 31 August 2018

ER -

Radulovic M, Asifuzzaman K, Carpenter P, Radojković P, Ayguadé E. HPC Benchmarking: Scaling Right and Looking Beyond the Average. In Torquati M, Aldinucci M, Padovani L, editors, Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings. Springer Verlag. 2018. p. 135-146. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-96983-1_10

HPC Benchmarking: Scaling Right and Looking Beyond the Average

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this