Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System

Sri Raj Paul; Akihiro Hayashi; Matthew Whitlock; Seonmyeong Bak; Keita Teranishi; Jackson Mayo; Max Grossman; Vivek Sarkar

doi:10.1109/ExaMPI52011.2020.00010

Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System

Sri Raj Paul, Akihiro Hayashi, Matthew Whitlock, Seonmyeong Bak, Keita Teranishi, Jackson Mayo, Max Grossman, Vivek Sarkar

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Achieving fault tolerance is one of the significant challenges of exascale computing due to projected increases in soft/transient failures. While past work on software-based resilience techniques typically focused on traditional bulk-synchronous parallel programming models, we believe that Asynchronous Many-Task (AMT) programming models are better suited to enabling resiliency since they provide explicit abstractions of data and tasks which contribute to increased asynchrony and latency tolerance. In this paper, we extend our past work on enabling application-level resilience in single node AMT programs by integrating the capability to perform asynchronous MPI communication, thereby enabling resiliency across multiple nodes. We also enable resilience against fail-stop errors where our runtime will manage all re-execution of tasks and communication without user intervention. Our results show that we are able to add communication operations to resilient programs with low overhead, by offloading communication to dedicated communication workers and also recover from fail-stop errors transparently, thereby enhancing productivity.

Original language	English
Title of host publication	Proceedings of ExaMPI 2020
Subtitle of host publication	Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	41-51
Number of pages	11
ISBN (Electronic)	9781665415613
DOIs	https://doi.org/10.1109/ExaMPI52011.2020.00010
State	Published - Nov 2020
Externally published	Yes
Event	2020 Exascale MPI Workshop, ExaMPI 2020 - Virtual, Atlanta, United States Duration: Nov 13 2020 → …

Publication series

Name	Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	2020 Exascale MPI Workshop, ExaMPI 2020
Country/Territory	United States
City	Virtual, Atlanta
Period	11/13/20 → …

Funding

Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration (NNSA) under contract DE-NA0003525. This work was funded by NNSA’s Advanced Simulation and Computing (ASC) Program. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility operated under Contract No. DE-AC02-05CH11231.

Keywords

AMT Runtimes
Fenix
Habanero C/C++
MPI communication
MPI-ULFM
Resilience

Access to Document

10.1109/ExaMPI52011.2020.00010

Cite this

Paul, S. R., Hayashi, A., Whitlock, M., Bak, S., Teranishi, K., Mayo, J., Grossman, M., & Sarkar, V. (2020). Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System. In Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 41-51). Article 9308655 (Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ExaMPI52011.2020.00010

Paul, Sri Raj ; Hayashi, Akihiro ; Whitlock, Matthew et al. / Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System. Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 41-51 (Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{3ca82e739d474c9c8b143d8c9ca68487,

title = "Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System",

abstract = "Achieving fault tolerance is one of the significant challenges of exascale computing due to projected increases in soft/transient failures. While past work on software-based resilience techniques typically focused on traditional bulk-synchronous parallel programming models, we believe that Asynchronous Many-Task (AMT) programming models are better suited to enabling resiliency since they provide explicit abstractions of data and tasks which contribute to increased asynchrony and latency tolerance. In this paper, we extend our past work on enabling application-level resilience in single node AMT programs by integrating the capability to perform asynchronous MPI communication, thereby enabling resiliency across multiple nodes. We also enable resilience against fail-stop errors where our runtime will manage all re-execution of tasks and communication without user intervention. Our results show that we are able to add communication operations to resilient programs with low overhead, by offloading communication to dedicated communication workers and also recover from fail-stop errors transparently, thereby enhancing productivity.",

keywords = "AMT Runtimes, Fenix, Habanero C/C++, MPI communication, MPI-ULFM, Resilience",

author = "Paul, \{Sri Raj\} and Akihiro Hayashi and Matthew Whitlock and Seonmyeong Bak and Keita Teranishi and Jackson Mayo and Max Grossman and Vivek Sarkar",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 Exascale MPI Workshop, ExaMPI 2020 ; Conference date: 13-11-2020",

year = "2020",

month = nov,

doi = "10.1109/ExaMPI52011.2020.00010",

language = "English",

series = "Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "41--51",

booktitle = "Proceedings of ExaMPI 2020",

}

Paul, SR, Hayashi, A, Whitlock, M, Bak, S, Teranishi, K, Mayo, J, Grossman, M & Sarkar, V 2020, Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System. in Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis., 9308655, Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 41-51, 2020 Exascale MPI Workshop, ExaMPI 2020, Virtual, Atlanta, United States, 11/13/20. https://doi.org/10.1109/ExaMPI52011.2020.00010

Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System. / Paul, Sri Raj; Hayashi, Akihiro; Whitlock, Matthew et al.
Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2020. p. 41-51 9308655 (Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System

AU - Paul, Sri Raj

AU - Hayashi, Akihiro

AU - Whitlock, Matthew

AU - Bak, Seonmyeong

AU - Teranishi, Keita

AU - Mayo, Jackson

AU - Grossman, Max

AU - Sarkar, Vivek

PY - 2020/11

Y1 - 2020/11

N2 - Achieving fault tolerance is one of the significant challenges of exascale computing due to projected increases in soft/transient failures. While past work on software-based resilience techniques typically focused on traditional bulk-synchronous parallel programming models, we believe that Asynchronous Many-Task (AMT) programming models are better suited to enabling resiliency since they provide explicit abstractions of data and tasks which contribute to increased asynchrony and latency tolerance. In this paper, we extend our past work on enabling application-level resilience in single node AMT programs by integrating the capability to perform asynchronous MPI communication, thereby enabling resiliency across multiple nodes. We also enable resilience against fail-stop errors where our runtime will manage all re-execution of tasks and communication without user intervention. Our results show that we are able to add communication operations to resilient programs with low overhead, by offloading communication to dedicated communication workers and also recover from fail-stop errors transparently, thereby enhancing productivity.

AB - Achieving fault tolerance is one of the significant challenges of exascale computing due to projected increases in soft/transient failures. While past work on software-based resilience techniques typically focused on traditional bulk-synchronous parallel programming models, we believe that Asynchronous Many-Task (AMT) programming models are better suited to enabling resiliency since they provide explicit abstractions of data and tasks which contribute to increased asynchrony and latency tolerance. In this paper, we extend our past work on enabling application-level resilience in single node AMT programs by integrating the capability to perform asynchronous MPI communication, thereby enabling resiliency across multiple nodes. We also enable resilience against fail-stop errors where our runtime will manage all re-execution of tasks and communication without user intervention. Our results show that we are able to add communication operations to resilient programs with low overhead, by offloading communication to dedicated communication workers and also recover from fail-stop errors transparently, thereby enhancing productivity.

KW - AMT Runtimes

KW - Fenix

KW - Habanero C/C++

KW - MPI communication

KW - MPI-ULFM

KW - Resilience

UR - https://www.scopus.com/pages/publications/85099752246

U2 - 10.1109/ExaMPI52011.2020.00010

DO - 10.1109/ExaMPI52011.2020.00010

M3 - Conference contribution

AN - SCOPUS:85099752246

T3 - Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 41

EP - 51

BT - Proceedings of ExaMPI 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 Exascale MPI Workshop, ExaMPI 2020

Y2 - 13 November 2020

ER -

Paul SR, Hayashi A, Whitlock M, Bak S, Teranishi K, Mayo J et al. Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System. In Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2020. p. 41-51. 9308655. (Proceedings of ExaMPI 2020: Exascale MPI Workshop, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/ExaMPI52011.2020.00010

Integrating Inter-Node Communication with a Resilient Asynchronous Many-Task Runtime System

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this