Towards a high performance implementation of MPI-IO on the lustre file system

Phillip Dickens; Jeremy Logan

doi:10.1007/978-3-540-88871-0_61

Towards a high performance implementation of MPI-IO on the lustre file system

Phillip Dickens, Jeremy Logan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

11 Scopus citations

Abstract

Lustre is becoming an increasingly important file system for large-scale computing clusters. The problem is that many data-intensive applications use MPI-IO for their I/O requirements, and it has been well documented that MPI-IO performs poorly in a Lustre file system environment. However, the reasons for such poor performance are not currently well understood. We believe that the primary reason for poor performance is that the assumptions underpinning most of the parallel I/O optimizations implemented in MPI-IO do not hold in a Lustre environment. Perhaps the most important assumption that appears to be incorrect is that optimal performance is obtained by performing large, contiguous I/O operations. Our research suggests that this is often the worst approach to take in a Lustre file system. In fact, we found that the best performance is sometimes achieved when each process performs a series of smaller, non-contiguous I/O requests. In this paper, we provide experimental results showing that such assumptions do not apply in Lustre, and explore new approaches that appear to provide significantly better performance.

Original language	English
Title of host publication	On the Move to Meaningful Internet Systems
Subtitle of host publication	OTM 2008 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Proceedings
Pages	870-885
Number of pages	16
Edition	PART 1
DOIs	https://doi.org/10.1007/978-3-540-88871-0_61
State	Published - 2008
Externally published	Yes
Event	OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008 - Monterrey, Mexico Duration: Nov 9 2008 → Nov 14 2008

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 1
Volume	5331 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008
Country/Territory	Mexico
City	Monterrey
Period	11/9/08 → 11/14/08

Funding

This material is based upon work supported by the National Science Foundation under Grant No. 0702748.

Keywords

Collective I/O
Lustre
MPI-IO
Parallel File Systems
Parallel I/O

Access to Document

10.1007/978-3-540-88871-0_61

Cite this

Dickens, P., & Logan, J. (2008). Towards a high performance implementation of MPI-IO on the lustre file system. In On the Move to Meaningful Internet Systems: OTM 2008 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Proceedings (PART 1 ed., pp. 870-885). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5331 LNCS, No. PART 1). https://doi.org/10.1007/978-3-540-88871-0_61

Dickens, Phillip ; Logan, Jeremy. / Towards a high performance implementation of MPI-IO on the lustre file system. On the Move to Meaningful Internet Systems: OTM 2008 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Proceedings. PART 1. ed. 2008. pp. 870-885 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).

@inproceedings{360e35493dcb467ab3ff5da47888552a,

title = "Towards a high performance implementation of MPI-IO on the lustre file system",

abstract = "Lustre is becoming an increasingly important file system for large-scale computing clusters. The problem is that many data-intensive applications use MPI-IO for their I/O requirements, and it has been well documented that MPI-IO performs poorly in a Lustre file system environment. However, the reasons for such poor performance are not currently well understood. We believe that the primary reason for poor performance is that the assumptions underpinning most of the parallel I/O optimizations implemented in MPI-IO do not hold in a Lustre environment. Perhaps the most important assumption that appears to be incorrect is that optimal performance is obtained by performing large, contiguous I/O operations. Our research suggests that this is often the worst approach to take in a Lustre file system. In fact, we found that the best performance is sometimes achieved when each process performs a series of smaller, non-contiguous I/O requests. In this paper, we provide experimental results showing that such assumptions do not apply in Lustre, and explore new approaches that appear to provide significantly better performance.",

keywords = "Collective I/O, Lustre, MPI-IO, Parallel File Systems, Parallel I/O",

author = "Phillip Dickens and Jeremy Logan",

year = "2008",

doi = "10.1007/978-3-540-88871-0_61",

language = "English",

isbn = "3540888705",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "PART 1",

pages = "870--885",

booktitle = "On the Move to Meaningful Internet Systems",

edition = "PART 1",

note = "OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008 ; Conference date: 09-11-2008 Through 14-11-2008",

}

Dickens, P & Logan, J 2008, Towards a high performance implementation of MPI-IO on the lustre file system. in On the Move to Meaningful Internet Systems: OTM 2008 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Proceedings. PART 1 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 5331 LNCS, pp. 870-885, OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Monterrey, Mexico, 11/9/08. https://doi.org/10.1007/978-3-540-88871-0_61

Towards a high performance implementation of MPI-IO on the lustre file system. / Dickens, Phillip; Logan, Jeremy.
On the Move to Meaningful Internet Systems: OTM 2008 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Proceedings. PART 1. ed. 2008. p. 870-885 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5331 LNCS, No. PART 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Towards a high performance implementation of MPI-IO on the lustre file system

AU - Dickens, Phillip

AU - Logan, Jeremy

PY - 2008

Y1 - 2008

N2 - Lustre is becoming an increasingly important file system for large-scale computing clusters. The problem is that many data-intensive applications use MPI-IO for their I/O requirements, and it has been well documented that MPI-IO performs poorly in a Lustre file system environment. However, the reasons for such poor performance are not currently well understood. We believe that the primary reason for poor performance is that the assumptions underpinning most of the parallel I/O optimizations implemented in MPI-IO do not hold in a Lustre environment. Perhaps the most important assumption that appears to be incorrect is that optimal performance is obtained by performing large, contiguous I/O operations. Our research suggests that this is often the worst approach to take in a Lustre file system. In fact, we found that the best performance is sometimes achieved when each process performs a series of smaller, non-contiguous I/O requests. In this paper, we provide experimental results showing that such assumptions do not apply in Lustre, and explore new approaches that appear to provide significantly better performance.

AB - Lustre is becoming an increasingly important file system for large-scale computing clusters. The problem is that many data-intensive applications use MPI-IO for their I/O requirements, and it has been well documented that MPI-IO performs poorly in a Lustre file system environment. However, the reasons for such poor performance are not currently well understood. We believe that the primary reason for poor performance is that the assumptions underpinning most of the parallel I/O optimizations implemented in MPI-IO do not hold in a Lustre environment. Perhaps the most important assumption that appears to be incorrect is that optimal performance is obtained by performing large, contiguous I/O operations. Our research suggests that this is often the worst approach to take in a Lustre file system. In fact, we found that the best performance is sometimes achieved when each process performs a series of smaller, non-contiguous I/O requests. In this paper, we provide experimental results showing that such assumptions do not apply in Lustre, and explore new approaches that appear to provide significantly better performance.

KW - Collective I/O

KW - Lustre

KW - MPI-IO

KW - Parallel File Systems

KW - Parallel I/O

UR - http://www.scopus.com/inward/record.url?scp=58049111206&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-88871-0_61

DO - 10.1007/978-3-540-88871-0_61

M3 - Conference contribution

AN - SCOPUS:58049111206

SN - 3540888705

SN - 9783540888703

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 870

EP - 885

BT - On the Move to Meaningful Internet Systems

T2 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008

Y2 - 9 November 2008 through 14 November 2008

ER -

Dickens P, Logan J. Towards a high performance implementation of MPI-IO on the lustre file system. In On the Move to Meaningful Internet Systems: OTM 2008 - OTM 2008 Confederated International Conferences CoopIS, DOA, GADA, IS, and ODBASE 2008, Proceedings. PART 1 ed. 2008. p. 870-885. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). doi: 10.1007/978-3-540-88871-0_61

Towards a high performance implementation of MPI-IO on the lustre file system

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this