Optimizing I/O forwarding techniques for extreme-scale event tracing

Thomas Ilsche; Joseph Schuchart; Jason Cope; Dries Kimpe; Terry Jones; Andreas Knüpfer; Kamil Iskra; Robert Ross; Wolfgang E. Nagel; Stephen Poole

doi:10.1007/s10586-013-0272-9

Optimizing I/O forwarding techniques for extreme-scale event tracing

Thomas Ilsche, Joseph Schuchart, Jason Cope, Dries Kimpe, Terry Jones, Andreas Knüpfer, Kamil Iskra, Robert Ross, Wolfgang E. Nagel, Stephen Poole

Intelligent Systems and Facilities

Research output: Contribution to journal › Article › peer-review

4 Scopus citations

Abstract

Programming development tools are a vital component for understanding the behavior of parallel applications. Event tracing is a principal ingredient to these tools, but new and serious challenges place event tracing at risk on extreme-scale machines. As the quantity of captured events increases with concurrency, the additional data can overload the parallel file system and perturb the application being observed. In this work we present a solution for event tracing on extreme-scale machines. We enhance an I/O forwarding software layer to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system. Furthermore, we introduce a sophisticated write buffering capability to limit the impact. To validate the approach, we employ the Vampir tracing toolset using these new capabilities. Our results demonstrate that the approach increases the maximum traced application size by a factor of 5× to more than 200,000 processes.

Original language	English
Pages (from-to)	1-18
Number of pages	18
Journal	Cluster Computing
Volume	17
Issue number	1
DOIs	https://doi.org/10.1007/s10586-013-0272-9
State	Published - Mar 2014

Funding

Acknowledgements We thank Ramanan Sankaran (ORNL) for providing a working version of S3D as well as a benchmark problem set for JaguarPF. We are grateful to Matthias Jurenz for his assistance on VampirTrace as well as Matthias Weber and Ronald Geisler for their support for Vampir. The IOFSL project is supported by the DOE Office of Science and National Nuclear Security Administration (NNSA). This research used resources of the Argonne Leadership Computing Facility at Argonne National Laboratory and the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory, which are supported by the Office of Science of the U.S. Department of Energy under contracts DE-AC02-06CH11357 and DE-AC05-00OR22725, respectively. This work was supported in part by the National Science Foundation (NSF) through NSF-0937928 and NSF-0724599. This work is supported in a part by the German Research Foundation (DFG) in the Collaborative Research Center 912 “Highly Adaptive Energy-Efficient Computing“.

Funders	Funder number
National Science Foundation	NSF-0937928, NSF-0724599
U.S. Department of Energy	DE-AC05-00OR22725, DE-AC02-06CH11357
Office of Science
National Nuclear Security Administration
Argonne National Laboratory
Deutsche Forschungsgemeinschaft

Keywords

Atomic append
Event tracing
I/O forwarding

Access to Document

10.1007/s10586-013-0272-9

Cite this

@article{1c0df07eca664bc5bd0bf7889a7fb48a,

title = "Optimizing I/O forwarding techniques for extreme-scale event tracing",

abstract = "Programming development tools are a vital component for understanding the behavior of parallel applications. Event tracing is a principal ingredient to these tools, but new and serious challenges place event tracing at risk on extreme-scale machines. As the quantity of captured events increases with concurrency, the additional data can overload the parallel file system and perturb the application being observed. In this work we present a solution for event tracing on extreme-scale machines. We enhance an I/O forwarding software layer to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system. Furthermore, we introduce a sophisticated write buffering capability to limit the impact. To validate the approach, we employ the Vampir tracing toolset using these new capabilities. Our results demonstrate that the approach increases the maximum traced application size by a factor of 5× to more than 200,000 processes.",

keywords = "Atomic append, Event tracing, I/O forwarding",

author = "Thomas Ilsche and Joseph Schuchart and Jason Cope and Dries Kimpe and Terry Jones and Andreas Kn{\"u}pfer and Kamil Iskra and Robert Ross and Nagel, {Wolfgang E.} and Stephen Poole",

year = "2014",

month = mar,

doi = "10.1007/s10586-013-0272-9",

language = "English",

volume = "17",

pages = "1--18",

journal = "Cluster Computing",

issn = "1386-7857",

publisher = "Springer",

number = "1",

}

TY - JOUR

T1 - Optimizing I/O forwarding techniques for extreme-scale event tracing

AU - Ilsche, Thomas

AU - Schuchart, Joseph

AU - Cope, Jason

AU - Kimpe, Dries

AU - Jones, Terry

AU - Knüpfer, Andreas

AU - Iskra, Kamil

AU - Ross, Robert

AU - Nagel, Wolfgang E.

AU - Poole, Stephen

PY - 2014/3

Y1 - 2014/3

N2 - Programming development tools are a vital component for understanding the behavior of parallel applications. Event tracing is a principal ingredient to these tools, but new and serious challenges place event tracing at risk on extreme-scale machines. As the quantity of captured events increases with concurrency, the additional data can overload the parallel file system and perturb the application being observed. In this work we present a solution for event tracing on extreme-scale machines. We enhance an I/O forwarding software layer to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system. Furthermore, we introduce a sophisticated write buffering capability to limit the impact. To validate the approach, we employ the Vampir tracing toolset using these new capabilities. Our results demonstrate that the approach increases the maximum traced application size by a factor of 5× to more than 200,000 processes.

AB - Programming development tools are a vital component for understanding the behavior of parallel applications. Event tracing is a principal ingredient to these tools, but new and serious challenges place event tracing at risk on extreme-scale machines. As the quantity of captured events increases with concurrency, the additional data can overload the parallel file system and perturb the application being observed. In this work we present a solution for event tracing on extreme-scale machines. We enhance an I/O forwarding software layer to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system. Furthermore, we introduce a sophisticated write buffering capability to limit the impact. To validate the approach, we employ the Vampir tracing toolset using these new capabilities. Our results demonstrate that the approach increases the maximum traced application size by a factor of 5× to more than 200,000 processes.

KW - Atomic append

KW - Event tracing

KW - I/O forwarding

UR - http://www.scopus.com/inward/record.url?scp=84896394482&partnerID=8YFLogxK

U2 - 10.1007/s10586-013-0272-9

DO - 10.1007/s10586-013-0272-9

M3 - Article

AN - SCOPUS:84896394482

SN - 1386-7857

VL - 17

SP - 1

EP - 18

JO - Cluster Computing

JF - Cluster Computing

IS - 1

ER -

Optimizing I/O forwarding techniques for extreme-scale event tracing

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this