TY - GEN
T1 - Extracting information ASAP!
AU - Abbasi, Hasan
AU - Eisenhauer, Greg
AU - Klasky, Scott
AU - Schwan, Karsten
AU - Wolf, Matthew
PY - 2010
Y1 - 2010
N2 - Designing I/O systems capable of scaling up to deal with the next generation of extreme scale scientific environments is a significant challenge. Scientific applications already strain the capabilities of current filesystems and storage systems. This work presents a middleware-based approach that generalizes from previous work on staging areas to focus more generally on staging resources. Exploiting the steady increase of the ratio of compute capability to I/O bandwidth, the EnStage middleware system allows for metadata characterization and I/O processing to occur when as where appropriate. This includes in reserved staging areas, buffered memory, and even in the writing processes' execution context. Using the EnStage extension to previous work, we find a 1.4% increase in runtime due to additional functionality resulted in I/O time in the staging area dropping to only 16% of the non-reduced output.
AB - Designing I/O systems capable of scaling up to deal with the next generation of extreme scale scientific environments is a significant challenge. Scientific applications already strain the capabilities of current filesystems and storage systems. This work presents a middleware-based approach that generalizes from previous work on staging areas to focus more generally on staging resources. Exploiting the steady increase of the ratio of compute capability to I/O bandwidth, the EnStage middleware system allows for metadata characterization and I/O processing to occur when as where appropriate. This includes in reserved staging areas, buffered memory, and even in the writing processes' execution context. Using the EnStage extension to previous work, we find a 1.4% increase in runtime due to additional functionality resulted in I/O time in the staging area dropping to only 16% of the non-reduced output.
UR - http://www.scopus.com/inward/record.url?scp=78651396278&partnerID=8YFLogxK
U2 - 10.1109/PDSW.2010.5668088
DO - 10.1109/PDSW.2010.5668088
M3 - Conference contribution
AN - SCOPUS:78651396278
SN - 9781424489121
T3 - Proceedings of the 2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis
BT - Proceedings of the 2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis
T2 - 2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis
Y2 - 15 November 2010 through 15 November 2010
ER -