TY - GEN
T1 - Empirical analysis of a large-scale hierarchical storage system
AU - Yu, Weikuan
AU - Oral, H. Sarp
AU - Canon, R. Shane
AU - Vetter, Jeffrey S.
AU - Sankaran, Ramanan
PY - 2008
Y1 - 2008
N2 - To prepare for future peta- or exa-scale computing, it is important to gain a good understanding on what impacts a hierarchical storage system would have on the performance of data-intensive applications, and accordingly, how to leverage its strengths and mitigate possible risks. To this aim, this paper adopts a user-level perspective to empirically reveal the implications of storage organization to parallel programs running on Jaguar at the Oak Ridge National Laboratory. We first describe the hierarchical configuration of Jaguar's storage system. Then we evaluate the performance of individual storage components. In addition, we examine the scalability of metadata- and data-intensive benchmarks over Jaguar. We have discovered that the file distribution pattern can impact the aggregated I/O bandwidth. Based on our analysis, we have demonstrated that it is possible to improve the scalability of a representative application S3D by as much as 15%.
AB - To prepare for future peta- or exa-scale computing, it is important to gain a good understanding on what impacts a hierarchical storage system would have on the performance of data-intensive applications, and accordingly, how to leverage its strengths and mitigate possible risks. To this aim, this paper adopts a user-level perspective to empirically reveal the implications of storage organization to parallel programs running on Jaguar at the Oak Ridge National Laboratory. We first describe the hierarchical configuration of Jaguar's storage system. Then we evaluate the performance of individual storage components. In addition, we examine the scalability of metadata- and data-intensive benchmarks over Jaguar. We have discovered that the file distribution pattern can impact the aggregated I/O bandwidth. Based on our analysis, we have demonstrated that it is possible to improve the scalability of a representative application S3D by as much as 15%.
UR - http://www.scopus.com/inward/record.url?scp=52049112145&partnerID=8YFLogxK
U2 - 10.1007/978-3-540-85451-7_15
DO - 10.1007/978-3-540-85451-7_15
M3 - Conference contribution
AN - SCOPUS:52049112145
SN - 3540854509
SN - 9783540854500
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 130
EP - 140
BT - Euro-Par 2008 Parallel Processing - 14th International Euro-Par Conference, Proceedings
T2 - 14th International Euro-Par Conference, Euro-Par 2008
Y2 - 26 August 2008 through 29 August 2008
ER -