Exploring the effect of noise on the performance benefit of nonblocking allreduce

Patrick Widener; Kurt B. Ferreira; Scott Levy; Torsten Hoefler

doi:10.1145/2642769.2642786

Exploring the effect of noise on the performance benefit of nonblocking allreduce

Patrick Widener, Kurt B. Ferreira, Scott Levy, Torsten Hoefler

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

Relaxed synchronization offers the potential of maintaining application scalability by allowing many processes to make independent progress when some processes suffer delays. Yet, the benefits of this approach in important parallel workloads have not been investigated in detail. In this paper, we use a validated simulation approach to explore the noise mitigation effects of nonblocking allreduce in workloads where allreduce is a major contributor to total execution time. Although a nonblocking allreduce is unlikely to provide significant benefit to applications in the low-OS-noise environments expected in next-generation HPC systems, we show that it can potentially improve application runtime with respect to other noise types.

Original language	English
Title of host publication	Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014
Publisher	Association for Computing Machinery
Pages	77-82
Number of pages	6
ISBN (Electronic)	9781450328753
DOIs	https://doi.org/10.1145/2642769.2642786
State	Published - Sep 9 2014
Externally published	Yes
Event	21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014 - Kyoto, Japan Duration: Sep 9 2014 → Sep 12 2014

Publication series

Name	ACM International Conference Proceeding Series
Volume	09-12-September-2014

Conference

Conference	21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014
Country/Territory	Japan
City	Kyoto
Period	09/9/14 → 09/12/14

Keywords

Collective operations
Nonblocking collectives
OS noise

Access to Document

10.1145/2642769.2642786

Cite this

Widener, P., Ferreira, K. B., Levy, S., & Hoefler, T. (2014). Exploring the effect of noise on the performance benefit of nonblocking allreduce. In Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014 (pp. 77-82). (ACM International Conference Proceeding Series; Vol. 09-12-September-2014). Association for Computing Machinery. https://doi.org/10.1145/2642769.2642786

@inproceedings{32682a7b71dc46ab8ed698978c79097c,

title = "Exploring the effect of noise on the performance benefit of nonblocking allreduce",

abstract = "Relaxed synchronization offers the potential of maintaining application scalability by allowing many processes to make independent progress when some processes suffer delays. Yet, the benefits of this approach in important parallel workloads have not been investigated in detail. In this paper, we use a validated simulation approach to explore the noise mitigation effects of nonblocking allreduce in workloads where allreduce is a major contributor to total execution time. Although a nonblocking allreduce is unlikely to provide significant benefit to applications in the low-OS-noise environments expected in next-generation HPC systems, we show that it can potentially improve application runtime with respect to other noise types.",

keywords = "Collective operations, Nonblocking collectives, OS noise",

author = "Patrick Widener and Ferreira, {Kurt B.} and Scott Levy and Torsten Hoefler",

note = "Publisher Copyright: {\textcopyright} ACM 2014.; 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014 ; Conference date: 09-09-2014 Through 12-09-2014",

year = "2014",

month = sep,

day = "9",

doi = "10.1145/2642769.2642786",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "77--82",

booktitle = "Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014",

}

Widener, P, Ferreira, KB, Levy, S & Hoefler, T 2014, Exploring the effect of noise on the performance benefit of nonblocking allreduce. in Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014. ACM International Conference Proceeding Series, vol. 09-12-September-2014, Association for Computing Machinery, pp. 77-82, 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014, Kyoto, Japan, 09/9/14. https://doi.org/10.1145/2642769.2642786

Exploring the effect of noise on the performance benefit of nonblocking allreduce. / Widener, Patrick; Ferreira, Kurt B.; Levy, Scott et al.
Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014. Association for Computing Machinery, 2014. p. 77-82 (ACM International Conference Proceeding Series; Vol. 09-12-September-2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Exploring the effect of noise on the performance benefit of nonblocking allreduce

AU - Widener, Patrick

AU - Ferreira, Kurt B.

AU - Levy, Scott

AU - Hoefler, Torsten

PY - 2014/9/9

Y1 - 2014/9/9

N2 - Relaxed synchronization offers the potential of maintaining application scalability by allowing many processes to make independent progress when some processes suffer delays. Yet, the benefits of this approach in important parallel workloads have not been investigated in detail. In this paper, we use a validated simulation approach to explore the noise mitigation effects of nonblocking allreduce in workloads where allreduce is a major contributor to total execution time. Although a nonblocking allreduce is unlikely to provide significant benefit to applications in the low-OS-noise environments expected in next-generation HPC systems, we show that it can potentially improve application runtime with respect to other noise types.

AB - Relaxed synchronization offers the potential of maintaining application scalability by allowing many processes to make independent progress when some processes suffer delays. Yet, the benefits of this approach in important parallel workloads have not been investigated in detail. In this paper, we use a validated simulation approach to explore the noise mitigation effects of nonblocking allreduce in workloads where allreduce is a major contributor to total execution time. Although a nonblocking allreduce is unlikely to provide significant benefit to applications in the low-OS-noise environments expected in next-generation HPC systems, we show that it can potentially improve application runtime with respect to other noise types.

KW - Collective operations

KW - Nonblocking collectives

KW - OS noise

UR - http://www.scopus.com/inward/record.url?scp=84958954568&partnerID=8YFLogxK

U2 - 10.1145/2642769.2642786

DO - 10.1145/2642769.2642786

M3 - Conference contribution

AN - SCOPUS:84958954568

T3 - ACM International Conference Proceeding Series

SP - 77

EP - 82

BT - Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014

PB - Association for Computing Machinery

T2 - 21st European MPI Users' Group Meeting, EuroMPI/ASIA 2014

Y2 - 9 September 2014 through 12 September 2014

ER -

Exploring the effect of noise on the performance benefit of nonblocking allreduce

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this