Accelerating Flash-X Simulations with Asynchronous I/O

Rajeev Jain; Houjun Tang; Akash Dhruv; J. Austin Harris; Suren Byna

doi:10.1109/PDSW56643.2022.00008

Accelerating Flash-X Simulations with Asynchronous I/O

Rajeev Jain, Houjun Tang, Akash Dhruv, J. Austin Harris, Suren Byna

Advanced Computing for Nuclear, Particle

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Most high-fidelity physics simulation codes, such as Flash-X, need to save intermediate results (checkpoint files) to restart or gain insights into the evolution of the simulation. These simulation codes save such intermediate files synchronously, where computation is stalled while the data is written to storage. Depending on the problem size and computational requirements, this file write time can be a substantial portion of the total simulation time. In order to hide the I/O latency of checkpointing, asynchronous I/O methods have been introduced. These methods use background threads for performing I/O while the main threads continue with the simulation. The usage of background threads can compete for resources on the node as well as with communication. In this paper, we evaluate the overheads and the overall benefit of asynchronous I/O in HDF5 to simulations. Results from real-world high-fidelity simulations on the Summit supercomputer show that I/O operation is overlapped with application communication or computation or both, effectively hiding some or all of the I/O latency. Our evaluation shows that while using asynchronous I/O adds overhead to the application, the I/O time reduction is more significant, resulting in overall up to 1.5X performance speedup.

Original language	English
Title of host publication	Proceedings of PDSW 2022
Subtitle of host publication	7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	13-19
Number of pages	7
ISBN (Electronic)	9781665475624
DOIs	https://doi.org/10.1109/PDSW56643.2022.00008
State	Published - 2022
Event	7th IEEE/ACM International Parallel Data Systems Workshop, PDSW 2022 - Dallas, United States Duration: Nov 13 2022 → Nov 18 2022

Publication series

Name	Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	7th IEEE/ACM International Parallel Data Systems Workshop, PDSW 2022
Country/Territory	United States
City	Dallas
Period	11/13/22 → 11/18/22

Funding

ACKNOWLEDGMENT This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaboratvi e effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract number DE-AC02-06CH11357.

Keywords

FlashX AsyncIO HDF5

Access to Document

10.1109/PDSW56643.2022.00008

Cite this

Jain, R., Tang, H., Dhruv, A., Harris, J. A., & Byna, S. (2022). Accelerating Flash-X Simulations with Asynchronous I/O. In Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 13-19). (Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PDSW56643.2022.00008

Jain, Rajeev ; Tang, Houjun ; Dhruv, Akash et al. / Accelerating Flash-X Simulations with Asynchronous I/O. Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 13-19 (Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{fce22357bf454df9963648270dab3af3,

title = "Accelerating Flash-X Simulations with Asynchronous I/O",

abstract = "Most high-fidelity physics simulation codes, such as Flash-X, need to save intermediate results (checkpoint files) to restart or gain insights into the evolution of the simulation. These simulation codes save such intermediate files synchronously, where computation is stalled while the data is written to storage. Depending on the problem size and computational requirements, this file write time can be a substantial portion of the total simulation time. In order to hide the I/O latency of checkpointing, asynchronous I/O methods have been introduced. These methods use background threads for performing I/O while the main threads continue with the simulation. The usage of background threads can compete for resources on the node as well as with communication. In this paper, we evaluate the overheads and the overall benefit of asynchronous I/O in HDF5 to simulations. Results from real-world high-fidelity simulations on the Summit supercomputer show that I/O operation is overlapped with application communication or computation or both, effectively hiding some or all of the I/O latency. Our evaluation shows that while using asynchronous I/O adds overhead to the application, the I/O time reduction is more significant, resulting in overall up to 1.5X performance speedup.",

keywords = "FlashX AsyncIO HDF5",

author = "Rajeev Jain and Houjun Tang and Akash Dhruv and Harris, {J. Austin} and Suren Byna",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 7th IEEE/ACM International Parallel Data Systems Workshop, PDSW 2022 ; Conference date: 13-11-2022 Through 18-11-2022",

year = "2022",

doi = "10.1109/PDSW56643.2022.00008",

language = "English",

series = "Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "13--19",

booktitle = "Proceedings of PDSW 2022",

}

Jain, R, Tang, H, Dhruv, A, Harris, JA & Byna, S 2022, Accelerating Flash-X Simulations with Asynchronous I/O. in Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 13-19, 7th IEEE/ACM International Parallel Data Systems Workshop, PDSW 2022, Dallas, United States, 11/13/22. https://doi.org/10.1109/PDSW56643.2022.00008

Accelerating Flash-X Simulations with Asynchronous I/O. / Jain, Rajeev; Tang, Houjun; Dhruv, Akash et al.
Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2022. p. 13-19 (Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Accelerating Flash-X Simulations with Asynchronous I/O

AU - Jain, Rajeev

AU - Tang, Houjun

AU - Dhruv, Akash

AU - Harris, J. Austin

AU - Byna, Suren

PY - 2022

Y1 - 2022

N2 - Most high-fidelity physics simulation codes, such as Flash-X, need to save intermediate results (checkpoint files) to restart or gain insights into the evolution of the simulation. These simulation codes save such intermediate files synchronously, where computation is stalled while the data is written to storage. Depending on the problem size and computational requirements, this file write time can be a substantial portion of the total simulation time. In order to hide the I/O latency of checkpointing, asynchronous I/O methods have been introduced. These methods use background threads for performing I/O while the main threads continue with the simulation. The usage of background threads can compete for resources on the node as well as with communication. In this paper, we evaluate the overheads and the overall benefit of asynchronous I/O in HDF5 to simulations. Results from real-world high-fidelity simulations on the Summit supercomputer show that I/O operation is overlapped with application communication or computation or both, effectively hiding some or all of the I/O latency. Our evaluation shows that while using asynchronous I/O adds overhead to the application, the I/O time reduction is more significant, resulting in overall up to 1.5X performance speedup.

AB - Most high-fidelity physics simulation codes, such as Flash-X, need to save intermediate results (checkpoint files) to restart or gain insights into the evolution of the simulation. These simulation codes save such intermediate files synchronously, where computation is stalled while the data is written to storage. Depending on the problem size and computational requirements, this file write time can be a substantial portion of the total simulation time. In order to hide the I/O latency of checkpointing, asynchronous I/O methods have been introduced. These methods use background threads for performing I/O while the main threads continue with the simulation. The usage of background threads can compete for resources on the node as well as with communication. In this paper, we evaluate the overheads and the overall benefit of asynchronous I/O in HDF5 to simulations. Results from real-world high-fidelity simulations on the Summit supercomputer show that I/O operation is overlapped with application communication or computation or both, effectively hiding some or all of the I/O latency. Our evaluation shows that while using asynchronous I/O adds overhead to the application, the I/O time reduction is more significant, resulting in overall up to 1.5X performance speedup.

KW - FlashX AsyncIO HDF5

UR - http://www.scopus.com/inward/record.url?scp=85147983497&partnerID=8YFLogxK

U2 - 10.1109/PDSW56643.2022.00008

DO - 10.1109/PDSW56643.2022.00008

M3 - Conference contribution

AN - SCOPUS:85147983497

T3 - Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 13

EP - 19

BT - Proceedings of PDSW 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 7th IEEE/ACM International Parallel Data Systems Workshop, PDSW 2022

Y2 - 13 November 2022 through 18 November 2022

ER -

Jain R, Tang H, Dhruv A, Harris JA, Byna S. Accelerating Flash-X Simulations with Asynchronous I/O. In Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2022. p. 13-19. (Proceedings of PDSW 2022: 7th International Parallel Data Systems Workshop, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/PDSW56643.2022.00008

Accelerating Flash-X Simulations with Asynchronous I/O

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this