Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer

Seung Hwan Lim; Ross G. Miller; Sudharshan S. Vazhkudai

doi:10.1109/IPDPS47924.2020.00028

Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer

Seung Hwan Lim, Ross G. Miller, Sudharshan S. Vazhkudai

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Designing dependable supercomputers begins with an understanding of errors in real-world, large-scale systems. The Titan supercomputer at Oak Ridge National Laboratory provides a unique opportunity to investigate errors when an actual system is actively used by multiple concurrent users and workloads from diverse domains at varying scales. This study presents a thorough analysis of 6, 908, 497 hardware errors from 18, 688 compute nodes of Titan for 312, 215 user jobs over a 3-year time period. Through careful joining of two system logs-the Machine Check Architecture (MCA) log and the job scheduler log-we show the correlated pattern of hardware errors for each job and user, in addition to individual descriptive statistics of errors, jobs, and users. Since the majority of hardware errors are memory errors, this study also shows the importance of error correcting in memory systems.

Original language	English
Title of host publication	Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	180-190
Number of pages	11
ISBN (Electronic)	9781728168760
DOIs	https://doi.org/10.1109/IPDPS47924.2020.00028
State	Published - May 2020
Event	34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020 - New Orleans, United States Duration: May 18 2020 → May 22 2020

Publication series

Name	Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020

Conference

Conference	34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020
Country/Territory	United States
City	New Orleans
Period	05/18/20 → 05/22/20

Funding

This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Funders	Funder number
U.S. Department of Energy

Access to Document

10.1109/IPDPS47924.2020.00028

Cite this

Lim, S. H., Miller, R. G., & Vazhkudai, S. S. (2020). Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer. In Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020 (pp. 180-190). Article 9139888 (Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS47924.2020.00028

Lim, Seung Hwan ; Miller, Ross G. ; Vazhkudai, Sudharshan S. / Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer. Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 180-190 (Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020).

@inproceedings{ba8888b4cc73407a89116956fef74722,

title = "Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer",

abstract = "Designing dependable supercomputers begins with an understanding of errors in real-world, large-scale systems. The Titan supercomputer at Oak Ridge National Laboratory provides a unique opportunity to investigate errors when an actual system is actively used by multiple concurrent users and workloads from diverse domains at varying scales. This study presents a thorough analysis of 6, 908, 497 hardware errors from 18, 688 compute nodes of Titan for 312, 215 user jobs over a 3-year time period. Through careful joining of two system logs-the Machine Check Architecture (MCA) log and the job scheduler log-we show the correlated pattern of hardware errors for each job and user, in addition to individual descriptive statistics of errors, jobs, and users. Since the majority of hardware errors are memory errors, this study also shows the importance of error correcting in memory systems.",

author = "Lim, {Seung Hwan} and Miller, {Ross G.} and Vazhkudai, {Sudharshan S.}",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020 ; Conference date: 18-05-2020 Through 22-05-2020",

year = "2020",

month = may,

doi = "10.1109/IPDPS47924.2020.00028",

language = "English",

series = "Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "180--190",

booktitle = "Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020",

}

Lim, SH , Miller, RG & Vazhkudai, SS 2020, Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer. in Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020., 9139888, Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020, Institute of Electrical and Electronics Engineers Inc., pp. 180-190, 34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020, New Orleans, United States, 05/18/20. https://doi.org/10.1109/IPDPS47924.2020.00028

Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer. / Lim, Seung Hwan ; Miller, Ross G.; Vazhkudai, Sudharshan S.
Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020. Institute of Electrical and Electronics Engineers Inc., 2020. p. 180-190 9139888 (Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer

AU - Lim, Seung Hwan

AU - Miller, Ross G.

AU - Vazhkudai, Sudharshan S.

PY - 2020/5

Y1 - 2020/5

N2 - Designing dependable supercomputers begins with an understanding of errors in real-world, large-scale systems. The Titan supercomputer at Oak Ridge National Laboratory provides a unique opportunity to investigate errors when an actual system is actively used by multiple concurrent users and workloads from diverse domains at varying scales. This study presents a thorough analysis of 6, 908, 497 hardware errors from 18, 688 compute nodes of Titan for 312, 215 user jobs over a 3-year time period. Through careful joining of two system logs-the Machine Check Architecture (MCA) log and the job scheduler log-we show the correlated pattern of hardware errors for each job and user, in addition to individual descriptive statistics of errors, jobs, and users. Since the majority of hardware errors are memory errors, this study also shows the importance of error correcting in memory systems.

AB - Designing dependable supercomputers begins with an understanding of errors in real-world, large-scale systems. The Titan supercomputer at Oak Ridge National Laboratory provides a unique opportunity to investigate errors when an actual system is actively used by multiple concurrent users and workloads from diverse domains at varying scales. This study presents a thorough analysis of 6, 908, 497 hardware errors from 18, 688 compute nodes of Titan for 312, 215 user jobs over a 3-year time period. Through careful joining of two system logs-the Machine Check Architecture (MCA) log and the job scheduler log-we show the correlated pattern of hardware errors for each job and user, in addition to individual descriptive statistics of errors, jobs, and users. Since the majority of hardware errors are memory errors, this study also shows the importance of error correcting in memory systems.

UR - http://www.scopus.com/inward/record.url?scp=85088897388&partnerID=8YFLogxK

U2 - 10.1109/IPDPS47924.2020.00028

DO - 10.1109/IPDPS47924.2020.00028

M3 - Conference contribution

AN - SCOPUS:85088897388

T3 - Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020

SP - 180

EP - 190

BT - Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020

Y2 - 18 May 2020 through 22 May 2020

ER -

Lim SH , Miller RG, Vazhkudai SS. Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer. In Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020. Institute of Electrical and Electronics Engineers Inc. 2020. p. 180-190. 9139888. (Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020). doi: 10.1109/IPDPS47924.2020.00028

Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this