Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems

Sarp Oral; James Simmons; Jason Hill; Dustin Leverman; Feiyi Wang; Matt Ezell; Ross Miller; Douglas Fuller; Raghul Gunasekaran; Youngjae Kim; Saurabh Gupta; Devesh Tiwari; Sudharshan S. Vazhkudai; James H. Rogers; David Dillow; Galen M. Shipman; Arthur S. Bland

doi:10.1109/SC.2014.23

Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems

Sarp Oral, James Simmons, Jason Hill, Dustin Leverman, Feiyi Wang, Matt Ezell, Ross Miller, Douglas Fuller, Raghul Gunasekaran, Youngjae Kim, Saurabh Gupta, Devesh Tiwari, Sudharshan S. Vazhkudai, James H. Rogers, David Dillow, Galen M. Shipman, Arthur S. Bland

Research output: Contribution to journal › Conference article › peer-review

43 Scopus citations

Abstract

The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.

Original language	English
Article number	7013005
Pages (from-to)	217-228
Number of pages	12
Journal	International Conference for High Performance Computing, Networking, Storage and Analysis, SC
Volume	2015-January
Issue number	January
DOIs	https://doi.org/10.1109/SC.2014.23
State	Published - Jan 16 2014
Event	International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014 - New Orleans, United States Duration: Nov 16 2014 → Nov 21 2014

Access to Document

10.1109/SC.2014.23

Cite this

Oral, S., Simmons, J., Hill, J., Leverman, D., Wang, F., Ezell, M., Miller, R., Fuller, D., Gunasekaran, R., Kim, Y., Gupta, S., Tiwari, D., Vazhkudai, S. S., Rogers, J. H., Dillow, D., Shipman, G. M., & Bland, A. S. (2014). Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems. International Conference for High Performance Computing, Networking, Storage and Analysis, SC, 2015-January(January), 217-228. Article 7013005. https://doi.org/10.1109/SC.2014.23

@article{35d5956769314a9384e9e77d73d70ada,

title = "Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems",

abstract = "The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.",

author = "Sarp Oral and James Simmons and Jason Hill and Dustin Leverman and Feiyi Wang and Matt Ezell and Ross Miller and Douglas Fuller and Raghul Gunasekaran and Youngjae Kim and Saurabh Gupta and Devesh Tiwari and Vazhkudai, {Sudharshan S.} and Rogers, {James H.} and David Dillow and Shipman, {Galen M.} and Bland, {Arthur S.}",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014 ; Conference date: 16-11-2014 Through 21-11-2014",

year = "2014",

month = jan,

day = "16",

doi = "10.1109/SC.2014.23",

language = "English",

volume = "2015-January",

pages = "217--228",

journal = "International Conference for High Performance Computing, Networking, Storage and Analysis, SC",

issn = "2167-4329",

number = "January",

}

Oral, S , Simmons, J , Hill, J , Leverman, D , Wang, F , Ezell, M , Miller, R, Fuller, D, Gunasekaran, R, Kim, Y, Gupta, S, Tiwari, D, Vazhkudai, SS, Rogers, JH, Dillow, D, Shipman, GM & Bland, AS 2014, 'Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems', International Conference for High Performance Computing, Networking, Storage and Analysis, SC, vol. 2015-January, no. January, 7013005, pp. 217-228. https://doi.org/10.1109/SC.2014.23

Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems. / Oral, Sarp ; Simmons, James ; Hill, Jason et al.
In: International Conference for High Performance Computing, Networking, Storage and Analysis, SC, Vol. 2015-January, No. January, 7013005, 16.01.2014, p. 217-228.

Research output: Contribution to journal › Conference article › peer-review

TY - JOUR

T1 - Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems

AU - Oral, Sarp

AU - Simmons, James

AU - Hill, Jason

AU - Leverman, Dustin

AU - Wang, Feiyi

AU - Ezell, Matt

AU - Miller, Ross

AU - Fuller, Douglas

AU - Gunasekaran, Raghul

AU - Kim, Youngjae

AU - Gupta, Saurabh

AU - Tiwari, Devesh

AU - Vazhkudai, Sudharshan S.

AU - Rogers, James H.

AU - Dillow, David

AU - Shipman, Galen M.

AU - Bland, Arthur S.

PY - 2014/1/16

Y1 - 2014/1/16

N2 - The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.

AB - The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.

UR - http://www.scopus.com/inward/record.url?scp=84936946893&partnerID=8YFLogxK

U2 - 10.1109/SC.2014.23

DO - 10.1109/SC.2014.23

M3 - Conference article

AN - SCOPUS:84936946893

SN - 2167-4329

VL - 2015-January

SP - 217

EP - 228

JO - International Conference for High Performance Computing, Networking, Storage and Analysis, SC

JF - International Conference for High Performance Computing, Networking, Storage and Analysis, SC

IS - January

M1 - 7013005

T2 - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014

Y2 - 16 November 2014 through 21 November 2014

ER -

Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems

Abstract

Access to Document

Other files and links

Fingerprint

Cite this