Interval based I/O: A new approach to providing high performance parallel I/O

Jeremy Logan, Phillip Dickens

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Providing scalable, high-performance parallel I/O for data-intensive computations is beset by a number of difficult challenges. The most often cited difficulties include the noncontiguous I/O patterns prominent in scientific codes, the lack of support for parallel I/O optimizations in POSIX, the high cost of providing strict file consistency semantics, and the cost of accessing storage devices over a network. We believe, however, that a more fundamental problem is the legacy view of a file as a linear sequence of bytes. To address this issue, we are developing a new approach to parallel I/O that is based on what we term intervals and interval files. This paper provides an overview of the interval-IO system and a set of benchmarks demonstrating the power of this new approach.

Original languageEnglish
Title of host publicationProceedings - 2011 International Conference on Parallel Processing Workshops, ICPPW 2011
Pages289-297
Number of pages9
DOIs
StatePublished - 2011
Event2011 International Conference on Parallel Processing Workshops, ICPPW 2011 - Taipei City, Taiwan, Province of China
Duration: Sep 13 2011Sep 16 2011

Publication series

NameProceedings of the International Conference on Parallel Processing Workshops
ISSN (Print)1530-2016

Conference

Conference2011 International Conference on Parallel Processing Workshops, ICPPW 2011
Country/TerritoryTaiwan, Province of China
CityTaipei City
Period09/13/1109/16/11

Keywords

  • Data intensive computing
  • High-performance computing
  • Interval I/O
  • MPI-IO
  • Parallel I/O

Fingerprint

Dive into the research topics of 'Interval based I/O: A new approach to providing high performance parallel I/O'. Together they form a unique fingerprint.

Cite this