TY - GEN
T1 - Collective I/O tuning using analytical and machine learning models
AU - Isaila, Florin
AU - Balaprakash, Prasanna
AU - Wild, Stefan M.
AU - Kimpe, Dries
AU - Latham, Rob
AU - Ross, Rob
AU - Hovland, Paul
N1 - Publisher Copyright:
© 2015 IEEE.
PY - 2015/10/26
Y1 - 2015/10/26
N2 - The optimization of parallel I/O has become challenging because of the increasing storage hierarchy, performance variability of shared storage systems, and the number of factors in the hardware and software stacks that impact performance. In this paper, we perform an in-depth study of the complexity involved in I/O autotuning and performance modeling, including the architecture, software stack, and noise. We propose a novel hybrid model combining analytical models for communication and storage operations and black-box models for the performance of the individual operations. The experimental results show that the hybrid approach performs significantly better and shows a higher robustness to noise than state-of-the-art machine learning approaches, at the cost of a higher modeling complexity.
AB - The optimization of parallel I/O has become challenging because of the increasing storage hierarchy, performance variability of shared storage systems, and the number of factors in the hardware and software stacks that impact performance. In this paper, we perform an in-depth study of the complexity involved in I/O autotuning and performance modeling, including the architecture, software stack, and noise. We propose a novel hybrid model combining analytical models for communication and storage operations and black-box models for the performance of the individual operations. The experimental results show that the hybrid approach performs significantly better and shows a higher robustness to noise than state-of-the-art machine learning approaches, at the cost of a higher modeling complexity.
KW - I/O performance modeling
KW - Model-based tuning
KW - Statistical and analytical performance models
UR - http://www.scopus.com/inward/record.url?scp=84959305163&partnerID=8YFLogxK
U2 - 10.1109/CLUSTER.2015.29
DO - 10.1109/CLUSTER.2015.29
M3 - Conference contribution
AN - SCOPUS:84959305163
T3 - Proceedings - IEEE International Conference on Cluster Computing, ICCC
SP - 128
EP - 137
BT - Proceedings - 2015 IEEE International Conference on Cluster Computing, CLUSTER 2015
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - IEEE International Conference on Cluster Computing, CLUSTER 2015
Y2 - 8 September 2015 through 11 September 2015
ER -