Efficient data-movement for lightweight I/O

Ron A. Oldfield; Patrick Widener; Arthur B. Maccabe; Lee Ward; Todd Kordenbrock

doi:10.1109/CLUSTR.2006.311897

Efficient data-movement for lightweight I/O

Ron A. Oldfield, Patrick Widener, Arthur B. Maccabe, Lee Ward, Todd Kordenbrock

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

25 Scopus citations

Abstract

Efficient data movement is an important part of any high-performance I/O system, but it is especially critical for the current and next-generation of massively parallel processing (MPP) systems. In this paper, we discuss how the scale, architecture, and organization of current and proposed MPP systems impact the design of the data-movement scheme for the I/O system. We also describe and analyze the approach used by the Lightweight File Systems (LWFS) project, and we compare that approach to more conventional data-movement protocols used by small and mid-range clusters. Our results indicate that the data-movement strategy used by LWFS clearly outperforms conventional data-movement protocols, particularly as data sizes increase.

Original language	English
Title of host publication	2006 IEEE International Conference on Cluster Computing, Cluster 2006
DOIs	https://doi.org/10.1109/CLUSTR.2006.311897
State	Published - 2006
Externally published	Yes
Event	2006 IEEE International Conference on Cluster Computing, Cluster 2006 - Barcelona, Spain Duration: Sep 25 2006 → Sep 28 2006

Publication series

Name	Proceedings - IEEE International Conference on Cluster Computing, ICCC
ISSN (Print)	1552-5244

Conference

Conference	2006 IEEE International Conference on Cluster Computing, Cluster 2006
Country/Territory	Spain
City	Barcelona
Period	09/25/06 → 09/28/06

Access to Document

10.1109/CLUSTR.2006.311897

Cite this

@inproceedings{3e76d91b5f72477a8db6a17f89ac23c5,

title = "Efficient data-movement for lightweight I/O",

abstract = "Efficient data movement is an important part of any high-performance I/O system, but it is especially critical for the current and next-generation of massively parallel processing (MPP) systems. In this paper, we discuss how the scale, architecture, and organization of current and proposed MPP systems impact the design of the data-movement scheme for the I/O system. We also describe and analyze the approach used by the Lightweight File Systems (LWFS) project, and we compare that approach to more conventional data-movement protocols used by small and mid-range clusters. Our results indicate that the data-movement strategy used by LWFS clearly outperforms conventional data-movement protocols, particularly as data sizes increase.",

author = "Oldfield, {Ron A.} and Patrick Widener and Maccabe, {Arthur B.} and Lee Ward and Todd Kordenbrock",

year = "2006",

doi = "10.1109/CLUSTR.2006.311897",

language = "English",

isbn = "1424403286",

series = "Proceedings - IEEE International Conference on Cluster Computing, ICCC",

booktitle = "2006 IEEE International Conference on Cluster Computing, Cluster 2006",

note = "2006 IEEE International Conference on Cluster Computing, Cluster 2006 ; Conference date: 25-09-2006 Through 28-09-2006",

}

Oldfield, RA, Widener, P, Maccabe, AB, Ward, L & Kordenbrock, T 2006, Efficient data-movement for lightweight I/O. in 2006 IEEE International Conference on Cluster Computing, Cluster 2006., 4100403, Proceedings - IEEE International Conference on Cluster Computing, ICCC, 2006 IEEE International Conference on Cluster Computing, Cluster 2006, Barcelona, Spain, 09/25/06. https://doi.org/10.1109/CLUSTR.2006.311897

Efficient data-movement for lightweight I/O. / Oldfield, Ron A.; Widener, Patrick; Maccabe, Arthur B. et al.
2006 IEEE International Conference on Cluster Computing, Cluster 2006. 2006. 4100403 (Proceedings - IEEE International Conference on Cluster Computing, ICCC).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient data-movement for lightweight I/O

AU - Oldfield, Ron A.

AU - Widener, Patrick

AU - Maccabe, Arthur B.

AU - Ward, Lee

AU - Kordenbrock, Todd

PY - 2006

Y1 - 2006

N2 - Efficient data movement is an important part of any high-performance I/O system, but it is especially critical for the current and next-generation of massively parallel processing (MPP) systems. In this paper, we discuss how the scale, architecture, and organization of current and proposed MPP systems impact the design of the data-movement scheme for the I/O system. We also describe and analyze the approach used by the Lightweight File Systems (LWFS) project, and we compare that approach to more conventional data-movement protocols used by small and mid-range clusters. Our results indicate that the data-movement strategy used by LWFS clearly outperforms conventional data-movement protocols, particularly as data sizes increase.

AB - Efficient data movement is an important part of any high-performance I/O system, but it is especially critical for the current and next-generation of massively parallel processing (MPP) systems. In this paper, we discuss how the scale, architecture, and organization of current and proposed MPP systems impact the design of the data-movement scheme for the I/O system. We also describe and analyze the approach used by the Lightweight File Systems (LWFS) project, and we compare that approach to more conventional data-movement protocols used by small and mid-range clusters. Our results indicate that the data-movement strategy used by LWFS clearly outperforms conventional data-movement protocols, particularly as data sizes increase.

UR - http://www.scopus.com/inward/record.url?scp=46049083770&partnerID=8YFLogxK

U2 - 10.1109/CLUSTR.2006.311897

DO - 10.1109/CLUSTR.2006.311897

M3 - Conference contribution

AN - SCOPUS:46049083770

SN - 1424403286

SN - 9781424403288

T3 - Proceedings - IEEE International Conference on Cluster Computing, ICCC

BT - 2006 IEEE International Conference on Cluster Computing, Cluster 2006

T2 - 2006 IEEE International Conference on Cluster Computing, Cluster 2006

Y2 - 25 September 2006 through 28 September 2006

ER -

Efficient data-movement for lightweight I/O

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this