TY - GEN
T1 - Analysis and Modeling of the End-to-End I/O Performance on OLCF's Titan Supercomputer
AU - Wan, Lipeng
AU - Wolf, Matthew
AU - Wang, Feiyi
AU - Choi, Jong Youl
AU - Ostrouchov, George
AU - Klasky, Scott
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/7/2
Y1 - 2017/7/2
N2 - With the increase of scale and complexity seen in a variety of leadership-class scientific computation and simulation applications, it has become more important to understand their I/O performance characteristics. The user-observed performance is a combination of properties of how the application is using the HPC facility, as well as how others' use of the facility causes variability in the static machine capabilities. Our work leverages statistical analysis of I/O performance data gathered with fine time resolution over a full week from Titan supercomputer. Based on observed properties of the distribution of I/O latencies, we build a three-state hidden Markov model (HMM) to characterize the end-to-end I/O performance on Titan. We parameterize our model using part of the field-gathered I/O performance data and validate it against the rest. The validation results demonstrate that our model can capture the dynamics of end-to-end I/O performance on Titan accurately.
AB - With the increase of scale and complexity seen in a variety of leadership-class scientific computation and simulation applications, it has become more important to understand their I/O performance characteristics. The user-observed performance is a combination of properties of how the application is using the HPC facility, as well as how others' use of the facility causes variability in the static machine capabilities. Our work leverages statistical analysis of I/O performance data gathered with fine time resolution over a full week from Titan supercomputer. Based on observed properties of the distribution of I/O latencies, we build a three-state hidden Markov model (HMM) to characterize the end-to-end I/O performance on Titan. We parameterize our model using part of the field-gathered I/O performance data and validate it against the rest. The validation results demonstrate that our model can capture the dynamics of end-to-end I/O performance on Titan accurately.
UR - http://www.scopus.com/inward/record.url?scp=85047605937&partnerID=8YFLogxK
U2 - 10.1109/HPCC-SmartCity-DSS.2017.1
DO - 10.1109/HPCC-SmartCity-DSS.2017.1
M3 - Conference contribution
AN - SCOPUS:85047605937
T3 - Proceedings - 2017 IEEE 19th Intl Conference on High Performance Computing and Communications, HPCC 2017, 2017 IEEE 15th Intl Conference on Smart City, SmartCity 2017 and 2017 IEEE 3rd Intl Conference on Data Science and Systems, DSS 2017
SP - 1
EP - 9
BT - Proceedings - 2017 IEEE 19th Intl Conference on High Performance Computing and Communications, HPCC 2017, 2017 IEEE 15th Intl Conference on Smart City, SmartCity 2017 and 2017 IEEE 3rd Intl Conference on Data Science and Systems, DSS 2017
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 19th IEEE Intl Conference on High Performance Computing and Communications, 15th IEEE Intl Conference on Smart City, and 3rd IEEE Intl Conference on Data Science and Systems, HPCC/SmartCity/DSS 2017
Y2 - 18 December 2017 through 20 December 2017
ER -