Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime

Yoav Tock; Benjamin Mandler; José Moreira; Terry Jones

doi:10.1007/978-3-642-40047-6_37

Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime

Yoav Tock, Benjamin Mandler, José Moreira, Terry Jones

Intelligent Systems and Facilities

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

As HPC systems and applications get bigger and more complex, we are approaching an era in which resiliency and run-time elasticity concerns become paramount. We offer a building block for an alternative resiliency approach in which computations will be able to make progress while components fail, in addition to enabling a dynamic set of nodes throughout a computation lifetime. The core of our solution is a hierarchical scalable membership service providing eventual consistency semantics. An attribute replication service is used for hierarchy organization, and is exposed to external applications. Our solution is based on P2P technologies and provides resiliency and elastic runtime support at ultra large scales. Resulting middleware is general purpose while exploiting HPC platform unique features and architecture. We have implemented and tested this system on BlueGene/P with Linux, and using worst-case analysis, evaluated the service scalability as effective for up to 1M nodes.

Original language	English
Title of host publication	Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings
Pages	354-366
Number of pages	13
DOIs	https://doi.org/10.1007/978-3-642-40047-6_37
State	Published - 2013
Event	19th International Conference on Parallel Processing, Euro-Par 2013 - Aachen, Germany Duration: Aug 26 2013 → Aug 30 2013

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8097 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Parallel Processing, Euro-Par 2013
Country/Territory	Germany
City	Aachen
Period	08/26/13 → 08/30/13

Funding

This research received funding from the U.S. DoE under award No. DE-SC0002107; the European Community’s FP7/2007-2013 Programme under grant agreement No. 317862; and used resources of the Oak Ridge Leadership Computing Facility at ORNL, which is supported by the U.S. DoE Office of Science under Contract No. DE-AC05-00OR22725.

Access to Document

10.1007/978-3-642-40047-6_37

Cite this

Tock, Y., Mandler, B., Moreira, J., & Jones, T. (2013). Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime. In Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings (pp. 354-366). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8097 LNCS). https://doi.org/10.1007/978-3-642-40047-6_37

Tock, Yoav ; Mandler, Benjamin ; Moreira, José et al. / Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime. Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings. 2013. pp. 354-366 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{622bc74910f146b3909c7dc94d824346,

title = "Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime",

abstract = "As HPC systems and applications get bigger and more complex, we are approaching an era in which resiliency and run-time elasticity concerns become paramount. We offer a building block for an alternative resiliency approach in which computations will be able to make progress while components fail, in addition to enabling a dynamic set of nodes throughout a computation lifetime. The core of our solution is a hierarchical scalable membership service providing eventual consistency semantics. An attribute replication service is used for hierarchy organization, and is exposed to external applications. Our solution is based on P2P technologies and provides resiliency and elastic runtime support at ultra large scales. Resulting middleware is general purpose while exploiting HPC platform unique features and architecture. We have implemented and tested this system on BlueGene/P with Linux, and using worst-case analysis, evaluated the service scalability as effective for up to 1M nodes.",

author = "Yoav Tock and Benjamin Mandler and Jos{\'e} Moreira and Terry Jones",

year = "2013",

doi = "10.1007/978-3-642-40047-6_37",

language = "English",

isbn = "9783642400469",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "354--366",

booktitle = "Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings",

note = "19th International Conference on Parallel Processing, Euro-Par 2013 ; Conference date: 26-08-2013 Through 30-08-2013",

}

Tock, Y, Mandler, B, Moreira, J & Jones, T 2013, Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime. in Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8097 LNCS, pp. 354-366, 19th International Conference on Parallel Processing, Euro-Par 2013, Aachen, Germany, 08/26/13. https://doi.org/10.1007/978-3-642-40047-6_37

Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime. / Tock, Yoav; Mandler, Benjamin; Moreira, José et al.
Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings. 2013. p. 354-366 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8097 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime

AU - Tock, Yoav

AU - Mandler, Benjamin

AU - Moreira, José

AU - Jones, Terry

PY - 2013

Y1 - 2013

N2 - As HPC systems and applications get bigger and more complex, we are approaching an era in which resiliency and run-time elasticity concerns become paramount. We offer a building block for an alternative resiliency approach in which computations will be able to make progress while components fail, in addition to enabling a dynamic set of nodes throughout a computation lifetime. The core of our solution is a hierarchical scalable membership service providing eventual consistency semantics. An attribute replication service is used for hierarchy organization, and is exposed to external applications. Our solution is based on P2P technologies and provides resiliency and elastic runtime support at ultra large scales. Resulting middleware is general purpose while exploiting HPC platform unique features and architecture. We have implemented and tested this system on BlueGene/P with Linux, and using worst-case analysis, evaluated the service scalability as effective for up to 1M nodes.

AB - As HPC systems and applications get bigger and more complex, we are approaching an era in which resiliency and run-time elasticity concerns become paramount. We offer a building block for an alternative resiliency approach in which computations will be able to make progress while components fail, in addition to enabling a dynamic set of nodes throughout a computation lifetime. The core of our solution is a hierarchical scalable membership service providing eventual consistency semantics. An attribute replication service is used for hierarchy organization, and is exposed to external applications. Our solution is based on P2P technologies and provides resiliency and elastic runtime support at ultra large scales. Resulting middleware is general purpose while exploiting HPC platform unique features and architecture. We have implemented and tested this system on BlueGene/P with Linux, and using worst-case analysis, evaluated the service scalability as effective for up to 1M nodes.

UR - http://www.scopus.com/inward/record.url?scp=84883146961&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-40047-6_37

DO - 10.1007/978-3-642-40047-6_37

M3 - Conference contribution

AN - SCOPUS:84883146961

SN - 9783642400469

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 354

EP - 366

BT - Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings

T2 - 19th International Conference on Parallel Processing, Euro-Par 2013

Y2 - 26 August 2013 through 30 August 2013

ER -

Tock Y, Mandler B, Moreira J, Jones T. Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime. In Euro-Par 2013 Parallel Processing - 19th International Conference, Proceedings. 2013. p. 354-366. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-40047-6_37

Design and implementation of a scalable membership service for supercomputer resiliency-aware runtime

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this