Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching

Ahmad Maroof Karimi; Arnab K. Paul; Jong Youl Choi; Lipeng Wan; Feiyi Wang

doi:10.1109/MASCOTS59514.2023.10387610

Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching

Ahmad Maroof Karimi, Arnab K. Paul, Jong Youl Choi, Lipeng Wan, Feiyi Wang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper explores the potential opportunities for implementing file prefetching techniques on large-scale high-performance computing (HPC) systems. Specifically, we investigate the file access patterns of various applications across multiple scientific domains using two years' worth of Darshan I/O traces obtained from the Summit supercomputer. We identify recurring trends and patterns which indicate that prefetching can be effectively leveraged to improve data access performance on HPC systems. This study serves as a valuable reference for system architects and developers in the HPC community, providing insights into the opportunities and challenges associated with enabling file prefetching on large-scale HPC systems.

Original language	English
Title of host publication	Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023
Publisher	IEEE Computer Society
ISBN (Electronic)	9798350319484
DOIs	https://doi.org/10.1109/MASCOTS59514.2023.10387610
State	Published - 2023
Event	31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023 - Stony Brook, United States Duration: Oct 16 2023 → Oct 18 2023

Publication series

Name	Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS
ISSN (Print)	1526-7539

Conference

Conference	31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023
Country/Territory	United States
City	Stony Brook
Period	10/16/23 → 10/18/23

Funding

ACKNOWLEDGMENT This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. Notice: This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (https://www.energy.gov/doe-public-access-plan).

Keywords

Darshan
I/O Prefetching
Parallel File System

Access to Document

10.1109/MASCOTS59514.2023.10387610

Cite this

Karimi, A. M., Paul, A. K., Choi, J. Y., Wan, L., & Wang, F. (2023). Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching. In Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023 (Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS). IEEE Computer Society. https://doi.org/10.1109/MASCOTS59514.2023.10387610

Karimi, Ahmad Maroof ; Paul, Arnab K. ; Choi, Jong Youl et al. / Analyzing File Access Patterns on Large-Scale HPC Systems : Opportunities for File Prefetching. Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023. IEEE Computer Society, 2023. (Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS).

@inproceedings{0c8d54cd21734e98b9ea4eef7442fca7,

title = "Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching",

abstract = "This paper explores the potential opportunities for implementing file prefetching techniques on large-scale high-performance computing (HPC) systems. Specifically, we investigate the file access patterns of various applications across multiple scientific domains using two years' worth of Darshan I/O traces obtained from the Summit supercomputer. We identify recurring trends and patterns which indicate that prefetching can be effectively leveraged to improve data access performance on HPC systems. This study serves as a valuable reference for system architects and developers in the HPC community, providing insights into the opportunities and challenges associated with enabling file prefetching on large-scale HPC systems.",

keywords = "Darshan, I/O Prefetching, Parallel File System",

author = "Karimi, {Ahmad Maroof} and Paul, {Arnab K.} and Choi, {Jong Youl} and Lipeng Wan and Feiyi Wang",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023 ; Conference date: 16-10-2023 Through 18-10-2023",

year = "2023",

doi = "10.1109/MASCOTS59514.2023.10387610",

language = "English",

series = "Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS",

publisher = "IEEE Computer Society",

booktitle = "Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023",

}

Karimi, AM, Paul, AK, Choi, JY, Wan, L & Wang, F 2023, Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching. in Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023. Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS, IEEE Computer Society, 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023, Stony Brook, United States, 10/16/23. https://doi.org/10.1109/MASCOTS59514.2023.10387610

Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching. / Karimi, Ahmad Maroof; Paul, Arnab K.; Choi, Jong Youl et al.
Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023. IEEE Computer Society, 2023. (Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Analyzing File Access Patterns on Large-Scale HPC Systems

T2 - 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023

AU - Karimi, Ahmad Maroof

AU - Paul, Arnab K.

AU - Choi, Jong Youl

AU - Wan, Lipeng

AU - Wang, Feiyi

PY - 2023

Y1 - 2023

N2 - This paper explores the potential opportunities for implementing file prefetching techniques on large-scale high-performance computing (HPC) systems. Specifically, we investigate the file access patterns of various applications across multiple scientific domains using two years' worth of Darshan I/O traces obtained from the Summit supercomputer. We identify recurring trends and patterns which indicate that prefetching can be effectively leveraged to improve data access performance on HPC systems. This study serves as a valuable reference for system architects and developers in the HPC community, providing insights into the opportunities and challenges associated with enabling file prefetching on large-scale HPC systems.

AB - This paper explores the potential opportunities for implementing file prefetching techniques on large-scale high-performance computing (HPC) systems. Specifically, we investigate the file access patterns of various applications across multiple scientific domains using two years' worth of Darshan I/O traces obtained from the Summit supercomputer. We identify recurring trends and patterns which indicate that prefetching can be effectively leveraged to improve data access performance on HPC systems. This study serves as a valuable reference for system architects and developers in the HPC community, providing insights into the opportunities and challenges associated with enabling file prefetching on large-scale HPC systems.

KW - Darshan

KW - I/O Prefetching

KW - Parallel File System

UR - http://www.scopus.com/inward/record.url?scp=85184522682&partnerID=8YFLogxK

U2 - 10.1109/MASCOTS59514.2023.10387610

DO - 10.1109/MASCOTS59514.2023.10387610

M3 - Conference contribution

AN - SCOPUS:85184522682

T3 - Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS

BT - Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023

PB - IEEE Computer Society

Y2 - 16 October 2023 through 18 October 2023

ER -

Karimi AM, Paul AK, Choi JY, Wan L, Wang F. Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching. In Proceedings - 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS 2023. IEEE Computer Society. 2023. (Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS). doi: 10.1109/MASCOTS59514.2023.10387610

Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this