A codesign framework for online data analysis and reduction

Kshitij Mehta; Ian Foster; Scott Klasky; Bryce Allen; Matthew Wolf; Jeremy Logan; Eric Suchyta; Jong Choi; Keichi Takahashi; Igor Yakushin; Todd Munson

doi:10.1109/WORKS49585.2019.00007

A codesign framework for online data analysis and reduction

Kshitij Mehta, Ian Foster, Scott Klasky, Bryce Allen, Matthew Wolf, Jeremy Logan, Eric Suchyta, Jong Choi, Keichi Takahashi, Igor Yakushin, Todd Munson

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Scopus citations

Abstract

In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diffusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory.

Original language	English
Title of host publication	Proceedings of WORKS 2019
Subtitle of host publication	14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	11-20
Number of pages	10
ISBN (Electronic)	9781728159973
DOIs	https://doi.org/10.1109/WORKS49585.2019.00007
State	Published - Nov 2019
Event	14th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2019 - Denver, United States Duration: Nov 17 2019 → …

Publication series

Name	Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	14th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2019
Country/Territory	United States
City	Denver
Period	11/17/19 → …

Funding

This research was supported in part by the Exascale Computing Project (17-SC-20-SC) of the U.S. Department of Energy (DOE), and by DOE’s Advanced Scientific Research Office (ASCR) under contract DE-AC02-06CH11357. Additionally, this research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory and of the National Energy Research Scientific Computing Center, which are supported by the Office of Science of the U.S. Department of Energy under Contract Numbers DE-AC05-00OR22725 and DE-AC02-05CH11231, respectively.

Keywords

Cheetah
Codar
Codesign
Exascale
In situ
Online
Reduction
Savanna
Summit
Workflows

Access to Document

10.1109/WORKS49585.2019.00007

Cite this

Mehta, K., Foster, I., Klasky, S., Allen, B., Wolf, M., Logan, J., Suchyta, E., Choi, J., Takahashi, K., Yakushin, I., & Munson, T. (2019). A codesign framework for online data analysis and reduction. In Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 11-20). Article 8943548 (Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WORKS49585.2019.00007

Mehta, Kshitij ; Foster, Ian ; Klasky, Scott et al. / A codesign framework for online data analysis and reduction. Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 11-20 (Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{bb1f7523a4184bfd9d8c69292dec5c2e,

title = "A codesign framework for online data analysis and reduction",

abstract = "In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diffusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory.",

keywords = "Cheetah, Codar, Codesign, Exascale, In situ, Online, Reduction, Savanna, Summit, Workflows",

author = "Kshitij Mehta and Ian Foster and Scott Klasky and Bryce Allen and Matthew Wolf and Jeremy Logan and Eric Suchyta and Jong Choi and Keichi Takahashi and Igor Yakushin and Todd Munson",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 14th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2019 ; Conference date: 17-11-2019",

year = "2019",

month = nov,

doi = "10.1109/WORKS49585.2019.00007",

language = "English",

series = "Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "11--20",

booktitle = "Proceedings of WORKS 2019",

}

Mehta, K, Foster, I, Klasky, S, Allen, B, Wolf, M, Logan, J , Suchyta, E , Choi, J, Takahashi, K, Yakushin, I & Munson, T 2019, A codesign framework for online data analysis and reduction. in Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis., 8943548, Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 11-20, 14th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2019, Denver, United States, 11/17/19. https://doi.org/10.1109/WORKS49585.2019.00007

A codesign framework for online data analysis and reduction. / Mehta, Kshitij; Foster, Ian; Klasky, Scott et al.
Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2019. p. 11-20 8943548 (Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A codesign framework for online data analysis and reduction

AU - Mehta, Kshitij

AU - Foster, Ian

AU - Klasky, Scott

AU - Allen, Bryce

AU - Wolf, Matthew

AU - Logan, Jeremy

AU - Suchyta, Eric

AU - Choi, Jong

AU - Takahashi, Keichi

AU - Yakushin, Igor

AU - Munson, Todd

PY - 2019/11

Y1 - 2019/11

N2 - In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diffusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory.

AB - In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diffusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory.

KW - Cheetah

KW - Codar

KW - Codesign

KW - Exascale

KW - In situ

KW - Online

KW - Reduction

KW - Savanna

KW - Summit

KW - Workflows

UR - http://www.scopus.com/inward/record.url?scp=85078198574&partnerID=8YFLogxK

U2 - 10.1109/WORKS49585.2019.00007

DO - 10.1109/WORKS49585.2019.00007

M3 - Conference contribution

AN - SCOPUS:85078198574

T3 - Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 11

EP - 20

BT - Proceedings of WORKS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 14th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2019

Y2 - 17 November 2019

ER -

Mehta K, Foster I, Klasky S, Allen B, Wolf M, Logan J et al. A codesign framework for online data analysis and reduction. In Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2019. p. 11-20. 8943548. (Proceedings of WORKS 2019: 14th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/WORKS49585.2019.00007

A codesign framework for online data analysis and reduction

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this