TY - GEN
T1 - Interval based I/O
T2 - 2011 International Conference on Parallel Processing Workshops, ICPPW 2011
AU - Logan, Jeremy
AU - Dickens, Phillip
PY - 2011
Y1 - 2011
N2 - Providing scalable, high-performance parallel I/O for data-intensive computations is beset by a number of difficult challenges. The most often cited difficulties include the noncontiguous I/O patterns prominent in scientific codes, the lack of support for parallel I/O optimizations in POSIX, the high cost of providing strict file consistency semantics, and the cost of accessing storage devices over a network. We believe, however, that a more fundamental problem is the legacy view of a file as a linear sequence of bytes. To address this issue, we are developing a new approach to parallel I/O that is based on what we term intervals and interval files. This paper provides an overview of the interval-IO system and a set of benchmarks demonstrating the power of this new approach.
AB - Providing scalable, high-performance parallel I/O for data-intensive computations is beset by a number of difficult challenges. The most often cited difficulties include the noncontiguous I/O patterns prominent in scientific codes, the lack of support for parallel I/O optimizations in POSIX, the high cost of providing strict file consistency semantics, and the cost of accessing storage devices over a network. We believe, however, that a more fundamental problem is the legacy view of a file as a linear sequence of bytes. To address this issue, we are developing a new approach to parallel I/O that is based on what we term intervals and interval files. This paper provides an overview of the interval-IO system and a set of benchmarks demonstrating the power of this new approach.
KW - Data intensive computing
KW - High-performance computing
KW - Interval I/O
KW - MPI-IO
KW - Parallel I/O
UR - http://www.scopus.com/inward/record.url?scp=80155167162&partnerID=8YFLogxK
U2 - 10.1109/ICPPW.2011.45
DO - 10.1109/ICPPW.2011.45
M3 - Conference contribution
AN - SCOPUS:80155167162
SN - 9780769545110
T3 - Proceedings of the International Conference on Parallel Processing Workshops
SP - 289
EP - 297
BT - Proceedings - 2011 International Conference on Parallel Processing Workshops, ICPPW 2011
Y2 - 13 September 2011 through 16 September 2011
ER -