TY - GEN
T1 - Profiling and improving I/O performance of a large-scale climate scientific application
AU - Liu, Zhuo
AU - Wang, Bin
AU - Wang, Teng
AU - Tian, Yuan
AU - Xu, Cong
AU - Wang, Yandong
AU - Yu, Weikuan
AU - Cruz, Carlos A.
AU - Zhou, Shujia
AU - Clune, Tom
AU - Klasky, Scott
PY - 2013
Y1 - 2013
N2 - Exascale computing systems are soon to emerge, which will pose great challenges on the huge gap between computing and I/O performance. Many large-scale scientific applications play an important role in our daily life. The huge amounts of data generated by such applications require highly parallel and efficient I/O management policies. In this paper, we adopt a mission-critical scientific application, GEOS-5, as a case to profile and analyze the communication and I/O issues that are preventing applications from fully utilizing the underlying parallel storage systems. Through in-detail architectural and experimental characterization, we observe that current legacy I/O schemes incur significant network communication overheads and are unable to fully parallelize the data access, thus degrading applications' I/O performance and scalability. To address these inefficiencies, we redesign its I/O framework along with a set of parallel I/O techniques to achieve high scalability and performance. Evaluation results on the NASA discover cluster show that our optimization of GEOS- 5 with ADIOS has led to significant performance improvements compared to the original GEOS-5 implementation.
AB - Exascale computing systems are soon to emerge, which will pose great challenges on the huge gap between computing and I/O performance. Many large-scale scientific applications play an important role in our daily life. The huge amounts of data generated by such applications require highly parallel and efficient I/O management policies. In this paper, we adopt a mission-critical scientific application, GEOS-5, as a case to profile and analyze the communication and I/O issues that are preventing applications from fully utilizing the underlying parallel storage systems. Through in-detail architectural and experimental characterization, we observe that current legacy I/O schemes incur significant network communication overheads and are unable to fully parallelize the data access, thus degrading applications' I/O performance and scalability. To address these inefficiencies, we redesign its I/O framework along with a set of parallel I/O techniques to achieve high scalability and performance. Evaluation results on the NASA discover cluster show that our optimization of GEOS- 5 with ADIOS has led to significant performance improvements compared to the original GEOS-5 implementation.
UR - http://www.scopus.com/inward/record.url?scp=84891388679&partnerID=8YFLogxK
U2 - 10.1109/ICCCN.2013.6614174
DO - 10.1109/ICCCN.2013.6614174
M3 - Conference contribution
AN - SCOPUS:84891388679
SN - 9781467357746
T3 - Proceedings - International Conference on Computer Communications and Networks, ICCCN
BT - 22nd International Conference on Computer Communications and Networks, ICCCN 2013 - Conference Proceedings
T2 - 2013 IEEE 2013 22nd International Conference on Computer Communication and Networks, ICCCN 2013
Y2 - 30 July 2013 through 2 August 2013
ER -