End-to-end I/O portfolio for the summit supercomputing ecosystem

Sarp Oral; Sudharshan S. Vazhkudai; Feiyi Wang; Christopher Zimmer; Christopher Brumgard; Jesse Hanley; George Markomanolis; Ross Miller; Dustin Leverman; Scott Atchley; Veronica Vergara Larrea

doi:10.1145/3295500.3356157

End-to-end I/O portfolio for the summit supercomputing ecosystem

Sarp Oral, Sudharshan S. Vazhkudai, Feiyi Wang, Christopher Zimmer, Christopher Brumgard, Jesse Hanley, George Markomanolis, Ross Miller, Dustin Leverman, Scott Atchley, Veronica Vergara Larrea

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

32 Scopus citations

Abstract

The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its ecosystem of analysis platforms is composed of two distinct layers, namely the in-system layer and the center-wide parallel file system layer (PFS), Spider 3. The in-system layer uses node-local SSDs and provides 26.7 TB/s for reads, 9.7 TB/s for writes, and 4.6 billion IOPS to Summit. The Spider 3 PFS layer uses IBM's Spectrum Scale and provides 2.5 TB/s and 2.6 million IOPS to Summit and other systems. While deploying them as two distinct layers was operationally efficient, it also presented usability challenges in terms of multiple mount points and lack of transparency in data movement. To address these challenges, we have developed novel end-to-end I/O solutions for the concerted use of the two storage layers. We present the I/O subsystem architecture, the end-to-end I/O solution space, their design considerations and our deployment experience.

Original language	English
Title of host publication	Proceedings of SC 2019
Subtitle of host publication	The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	IEEE Computer Society
ISBN (Electronic)	9781450362290
DOIs	https://doi.org/10.1145/3295500.3356157
State	Published - Nov 17 2019
Event	2019 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019 - Denver, United States Duration: Nov 17 2019 → Nov 22 2019

Publication series

Name	International Conference for High Performance Computing, Networking, Storage and Analysis, SC
ISSN (Print)	2167-4329
ISSN (Electronic)	2167-4337

Conference

Conference	2019 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019
Country/Territory	United States
City	Denver
Period	11/17/19 → 11/22/19

Funding

This work was performed under the auspices of the U.S. DOE by Oak Ridge Leadership Computing Facility at ORNL under contract DE-AC05-00OR22725. The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

High performance computing
Parallel IO
Parallel file systems
Performance bechmarking

Access to Document

10.1145/3295500.3356157

Cite this

Oral, S., Vazhkudai, S. S., Wang, F., Zimmer, C., Brumgard, C., Hanley, J., Markomanolis, G., Miller, R., Leverman, D., Atchley, S., & Larrea, V. V. (2019). End-to-end I/O portfolio for the summit supercomputing ecosystem. In Proceedings of SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis Article a63 (International Conference for High Performance Computing, Networking, Storage and Analysis, SC). IEEE Computer Society. https://doi.org/10.1145/3295500.3356157

Oral, Sarp ; Vazhkudai, Sudharshan S. ; Wang, Feiyi et al. / End-to-end I/O portfolio for the summit supercomputing ecosystem. Proceedings of SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, 2019. (International Conference for High Performance Computing, Networking, Storage and Analysis, SC).

@inproceedings{d7a7ce71621a4d3b8a9bfdaa6fc88f20,

title = "End-to-end I/O portfolio for the summit supercomputing ecosystem",

abstract = "The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its ecosystem of analysis platforms is composed of two distinct layers, namely the in-system layer and the center-wide parallel file system layer (PFS), Spider 3. The in-system layer uses node-local SSDs and provides 26.7 TB/s for reads, 9.7 TB/s for writes, and 4.6 billion IOPS to Summit. The Spider 3 PFS layer uses IBM's Spectrum Scale and provides 2.5 TB/s and 2.6 million IOPS to Summit and other systems. While deploying them as two distinct layers was operationally efficient, it also presented usability challenges in terms of multiple mount points and lack of transparency in data movement. To address these challenges, we have developed novel end-to-end I/O solutions for the concerted use of the two storage layers. We present the I/O subsystem architecture, the end-to-end I/O solution space, their design considerations and our deployment experience.",

keywords = "High performance computing, Parallel IO, Parallel file systems, Performance bechmarking",

author = "Sarp Oral and Vazhkudai, {Sudharshan S.} and Feiyi Wang and Christopher Zimmer and Christopher Brumgard and Jesse Hanley and George Markomanolis and Ross Miller and Dustin Leverman and Scott Atchley and Larrea, {Veronica Vergara}",

note = "Publisher Copyright: {\textcopyright} 2019 ACM.; 2019 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019 ; Conference date: 17-11-2019 Through 22-11-2019",

year = "2019",

month = nov,

day = "17",

doi = "10.1145/3295500.3356157",

language = "English",

series = "International Conference for High Performance Computing, Networking, Storage and Analysis, SC",

publisher = "IEEE Computer Society",

booktitle = "Proceedings of SC 2019",

}

Oral, S, Vazhkudai, SS, Wang, F , Zimmer, C , Brumgard, C , Hanley, J, Markomanolis, G, Miller, R , Leverman, D , Atchley, S & Larrea, VV 2019, End-to-end I/O portfolio for the summit supercomputing ecosystem. in Proceedings of SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis., a63, International Conference for High Performance Computing, Networking, Storage and Analysis, SC, IEEE Computer Society, 2019 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019, Denver, United States, 11/17/19. https://doi.org/10.1145/3295500.3356157

End-to-end I/O portfolio for the summit supercomputing ecosystem. / Oral, Sarp; Vazhkudai, Sudharshan S.; Wang, Feiyi et al.
Proceedings of SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, 2019. a63 (International Conference for High Performance Computing, Networking, Storage and Analysis, SC).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - End-to-end I/O portfolio for the summit supercomputing ecosystem

AU - Oral, Sarp

AU - Vazhkudai, Sudharshan S.

AU - Wang, Feiyi

AU - Zimmer, Christopher

AU - Brumgard, Christopher

AU - Hanley, Jesse

AU - Markomanolis, George

AU - Miller, Ross

AU - Leverman, Dustin

AU - Atchley, Scott

AU - Larrea, Veronica Vergara

PY - 2019/11/17

Y1 - 2019/11/17

N2 - The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its ecosystem of analysis platforms is composed of two distinct layers, namely the in-system layer and the center-wide parallel file system layer (PFS), Spider 3. The in-system layer uses node-local SSDs and provides 26.7 TB/s for reads, 9.7 TB/s for writes, and 4.6 billion IOPS to Summit. The Spider 3 PFS layer uses IBM's Spectrum Scale and provides 2.5 TB/s and 2.6 million IOPS to Summit and other systems. While deploying them as two distinct layers was operationally efficient, it also presented usability challenges in terms of multiple mount points and lack of transparency in data movement. To address these challenges, we have developed novel end-to-end I/O solutions for the concerted use of the two storage layers. We present the I/O subsystem architecture, the end-to-end I/O solution space, their design considerations and our deployment experience.

AB - The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its ecosystem of analysis platforms is composed of two distinct layers, namely the in-system layer and the center-wide parallel file system layer (PFS), Spider 3. The in-system layer uses node-local SSDs and provides 26.7 TB/s for reads, 9.7 TB/s for writes, and 4.6 billion IOPS to Summit. The Spider 3 PFS layer uses IBM's Spectrum Scale and provides 2.5 TB/s and 2.6 million IOPS to Summit and other systems. While deploying them as two distinct layers was operationally efficient, it also presented usability challenges in terms of multiple mount points and lack of transparency in data movement. To address these challenges, we have developed novel end-to-end I/O solutions for the concerted use of the two storage layers. We present the I/O subsystem architecture, the end-to-end I/O solution space, their design considerations and our deployment experience.

KW - High performance computing

KW - Parallel IO

KW - Parallel file systems

KW - Performance bechmarking

UR - http://www.scopus.com/inward/record.url?scp=85076181703&partnerID=8YFLogxK

U2 - 10.1145/3295500.3356157

DO - 10.1145/3295500.3356157

M3 - Conference contribution

AN - SCOPUS:85076181703

T3 - International Conference for High Performance Computing, Networking, Storage and Analysis, SC

BT - Proceedings of SC 2019

PB - IEEE Computer Society

T2 - 2019 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019

Y2 - 17 November 2019 through 22 November 2019

ER -

Oral S, Vazhkudai SS, Wang F , Zimmer C , Brumgard C , Hanley J et al. End-to-end I/O portfolio for the summit supercomputing ecosystem. In Proceedings of SC 2019: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society. 2019. a63. (International Conference for High Performance Computing, Networking, Storage and Analysis, SC). doi: 10.1145/3295500.3356157

End-to-end I/O portfolio for the summit supercomputing ecosystem

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this