On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective

Axel Huebl; René Widera; Felix Schmitt; Alexander Matthes; Norbert Podhorszki; Jong Youl Choi; Scott Klasky; Michael Bussmann

doi:10.1007/978-3-319-67630-2_2

On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective

Axel Huebl, René Widera, Felix Schmitt, Alexander Matthes, Norbert Podhorszki, Jong Youl Choi, Scott Klasky, Michael Bussmann

Workflow Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today’s and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.

Original language	English
Title of host publication	High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers
Editors	Rio Yokota, Julian M. Kunkel, Michela Taufer, John Shalf
Publisher	Springer Verlag
Pages	15-29
Number of pages	15
ISBN (Print)	9783319676296
DOIs	https://doi.org/10.1007/978-3-319-67630-2_2
State	Published - 2017
Event	32nd International Conference on High Performance Computing, ISC High Performance 2017 - Frankfurt, Germany Duration: Jun 18 2017 → Jun 22 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10524 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	32nd International Conference on High Performance Computing, ISC High Performance 2017
Country/Territory	Germany
City	Frankfurt
Period	06/18/17 → 06/22/17

Funding

This project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No. 654220. An award of computer time was provided by the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Access to Document

10.1007/978-3-319-67630-2_2

Cite this

Huebl, A., Widera, R., Schmitt, F., Matthes, A., Podhorszki, N., Choi, J. Y., Klasky, S., & Bussmann, M. (2017). On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective. In R. Yokota, J. M. Kunkel, M. Taufer, & J. Shalf (Eds.), High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers (pp. 15-29). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10524 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-67630-2_2

Huebl, Axel ; Widera, René ; Schmitt, Felix et al. / On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective. High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers. editor / Rio Yokota ; Julian M. Kunkel ; Michela Taufer ; John Shalf. Springer Verlag, 2017. pp. 15-29 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{880fd486972c4a9b95ea62b35ff3abe0,

title = "On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective",

abstract = "We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today{\textquoteright}s and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.",

author = "Axel Huebl and Ren{\'e} Widera and Felix Schmitt and Alexander Matthes and Norbert Podhorszki and Choi, {Jong Youl} and Scott Klasky and Michael Bussmann",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2017.; 32nd International Conference on High Performance Computing, ISC High Performance 2017 ; Conference date: 18-06-2017 Through 22-06-2017",

year = "2017",

doi = "10.1007/978-3-319-67630-2_2",

language = "English",

isbn = "9783319676296",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "15--29",

editor = "Rio Yokota and Kunkel, {Julian M.} and Michela Taufer and John Shalf",

booktitle = "High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers",

}

Huebl, A, Widera, R, Schmitt, F, Matthes, A, Podhorszki, N , Choi, JY , Klasky, S & Bussmann, M 2017, On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective. in R Yokota, JM Kunkel, M Taufer & J Shalf (eds), High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10524 LNCS, Springer Verlag, pp. 15-29, 32nd International Conference on High Performance Computing, ISC High Performance 2017, Frankfurt, Germany, 06/18/17. https://doi.org/10.1007/978-3-319-67630-2_2

On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective. / Huebl, Axel; Widera, René; Schmitt, Felix et al.
High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers. ed. / Rio Yokota; Julian M. Kunkel; Michela Taufer; John Shalf. Springer Verlag, 2017. p. 15-29 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10524 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective

AU - Huebl, Axel

AU - Widera, René

AU - Schmitt, Felix

AU - Matthes, Alexander

AU - Podhorszki, Norbert

AU - Choi, Jong Youl

AU - Klasky, Scott

AU - Bussmann, Michael

N1 - Publisher Copyright: © Springer International Publishing AG 2017.

PY - 2017

Y1 - 2017

N2 - We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today’s and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.

AB - We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today’s and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.

UR - http://www.scopus.com/inward/record.url?scp=85032655814&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-67630-2_2

DO - 10.1007/978-3-319-67630-2_2

M3 - Conference contribution

AN - SCOPUS:85032655814

SN - 9783319676296

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 15

EP - 29

BT - High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers

A2 - Yokota, Rio

A2 - Kunkel, Julian M.

A2 - Taufer, Michela

A2 - Shalf, John

PB - Springer Verlag

T2 - 32nd International Conference on High Performance Computing, ISC High Performance 2017

Y2 - 18 June 2017 through 22 June 2017

ER -

Huebl A, Widera R, Schmitt F, Matthes A, Podhorszki N , Choi JY et al. On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective. In Yokota R, Kunkel JM, Taufer M, Shalf J, editors, High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers. Springer Verlag. 2017. p. 15-29. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-67630-2_2

On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this