UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

Michael J. Brim; Adam T. Moody; Seung Hwan Lim; Ross Miller; Swen Boehm; Cameron Stanavige; Kathryn M. Mohror; Sarp Oral

doi:10.1109/IPDPS54959.2023.00037

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

Michael J. Brim, Adam T. Moody, Seung Hwan Lim, Ross Miller, Swen Boehm, Cameron Stanavige, Kathryn M. Mohror, Sarp Oral

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

14 Scopus citations

Abstract

We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.

Original language	English
Title of host publication	Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	290-300
Number of pages	11
ISBN (Electronic)	9798350337662
DOIs	https://doi.org/10.1109/IPDPS54959.2023.00037
State	Published - 2023
Event	37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 - St. Petersburg, United States Duration: May 15 2023 → May 19 2023

Publication series

Name	Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

Conference

Conference	37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023
Country/Territory	United States
City	St. Petersburg
Period	05/15/23 → 05/19/23

Funding

Notice: This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). ACKNOWLEDGMENTS This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaboratvi e effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.This work was performed under the auspices of the U.S. Department of Energy by Oak Ridge National Laboratory under Contract DE-AC05-00OR22725 and Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. LLNL-CONF-829824.

Keywords

Distributed file systems
Parallel I/O
Parallel systems
Storage devices
Storage hierarchies

Access to Document

10.1109/IPDPS54959.2023.00037

Cite this

Brim, M. J., Moody, A. T., Lim, S. H., Miller, R., Boehm, S., Stanavige, C., Mohror, K. M., & Oral, S. (2023). UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage. In Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 (pp. 290-300). (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS54959.2023.00037

Brim, Michael J. ; Moody, Adam T. ; Lim, Seung Hwan et al. / UnifyFS : A User-level Shared File System for Unified Access to Distributed Local Storage. Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 290-300 (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023).

@inproceedings{80ddf331272245f4bf09c86f5fdc7da9,

title = "UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage",

abstract = "We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.",

keywords = "Distributed file systems, Parallel I/O, Parallel systems, Storage devices, Storage hierarchies",

author = "Brim, {Michael J.} and Moody, {Adam T.} and Lim, {Seung Hwan} and Ross Miller and Swen Boehm and Cameron Stanavige and Mohror, {Kathryn M.} and Sarp Oral",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 ; Conference date: 15-05-2023 Through 19-05-2023",

year = "2023",

doi = "10.1109/IPDPS54959.2023.00037",

language = "English",

series = "Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "290--300",

booktitle = "Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023",

}

Brim, MJ, Moody, AT, Lim, SH , Miller, R , Boehm, S, Stanavige, C, Mohror, KM & Oral, S 2023, UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage. in Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023, Institute of Electrical and Electronics Engineers Inc., pp. 290-300, 37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023, St. Petersburg, United States, 05/15/23. https://doi.org/10.1109/IPDPS54959.2023.00037

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage. / Brim, Michael J.; Moody, Adam T.; Lim, Seung Hwan et al.
Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 290-300 (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - UnifyFS

T2 - 37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

AU - Brim, Michael J.

AU - Moody, Adam T.

AU - Lim, Seung Hwan

AU - Miller, Ross

AU - Boehm, Swen

AU - Stanavige, Cameron

AU - Mohror, Kathryn M.

AU - Oral, Sarp

PY - 2023

Y1 - 2023

N2 - We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.

AB - We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.

KW - Distributed file systems

KW - Parallel I/O

KW - Parallel systems

KW - Storage devices

KW - Storage hierarchies

UR - http://www.scopus.com/inward/record.url?scp=85166674106&partnerID=8YFLogxK

U2 - 10.1109/IPDPS54959.2023.00037

DO - 10.1109/IPDPS54959.2023.00037

M3 - Conference contribution

AN - SCOPUS:85166674106

T3 - Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

SP - 290

EP - 300

BT - Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 15 May 2023 through 19 May 2023

ER -

Brim MJ, Moody AT, Lim SH , Miller R , Boehm S, Stanavige C et al. UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage. In Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 290-300. (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023). doi: 10.1109/IPDPS54959.2023.00037

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this