TY - GEN
T1 - Battle of the defaults
T2 - 21st IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2021
AU - Xie, Bing
AU - Tang, Houjun
AU - Byna, Suren
AU - Hanley, Jesse
AU - Koziol, Quincey
AU - Li, Tonglin
AU - Oral, Sarp
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/5
Y1 - 2021/5
N2 - Popular parallel I/O libraries, such as HDF5, provide tuning parameters to obtain superior performance. However, the selection of effective parameters on production systems is complex due to the interdependence of I/O software and file system layers. Hence, application developers typically use the default parameters and often experience poor I/O performance. This work conducts a benchmarking-based analysis on the HDF5 behaviors with a wide variety of I/O patterns to extract performance characteristics under the production workload. To make the analysis well controlled, we exercise I/O benchmarks on POSIX-IO, MPI-IO, and HDF5 using the same I/O patterns and in the same jobs. To address high performance variability in production environments, we repeat the benchmarks across I/O patterns, storage devices, and time intervals. Based on the results, we identified consistent HDF5 behaviors that appropriate configurations and operations on dataset layout and file-metadata placement can improve performance significantly. We apply our findings and evaluate the tuned I/O library on two supercomputers: Summit and Cori. The results show that our tuned parameters can achieve more than 10× I/O performance speedup than that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.
AB - Popular parallel I/O libraries, such as HDF5, provide tuning parameters to obtain superior performance. However, the selection of effective parameters on production systems is complex due to the interdependence of I/O software and file system layers. Hence, application developers typically use the default parameters and often experience poor I/O performance. This work conducts a benchmarking-based analysis on the HDF5 behaviors with a wide variety of I/O patterns to extract performance characteristics under the production workload. To make the analysis well controlled, we exercise I/O benchmarks on POSIX-IO, MPI-IO, and HDF5 using the same I/O patterns and in the same jobs. To address high performance variability in production environments, we repeat the benchmarks across I/O patterns, storage devices, and time intervals. Based on the results, we identified consistent HDF5 behaviors that appropriate configurations and operations on dataset layout and file-metadata placement can improve performance significantly. We apply our findings and evaluate the tuned I/O library on two supercomputers: Summit and Cori. The results show that our tuned parameters can achieve more than 10× I/O performance speedup than that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.
UR - http://www.scopus.com/inward/record.url?scp=85114888893&partnerID=8YFLogxK
U2 - 10.1109/CCGrid51090.2021.00015
DO - 10.1109/CCGrid51090.2021.00015
M3 - Conference contribution
AN - SCOPUS:85114888893
T3 - Proceedings - 21st IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2021
SP - 51
EP - 60
BT - Proceedings - 21st IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2021
A2 - Lefevre, Laurent
A2 - Patterson, Stacy
A2 - Lee, Young Choon
A2 - Shen, Haiying
A2 - Ilager, Shashikant
A2 - Goudarzi, Mohammad
A2 - Toosi, Adel N.
A2 - Buyya, Rajkumar
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 10 May 2021 through 13 May 2021
ER -