OpenSHMEM sets and groups: An approach to worksharing and memory management

Ferrol Aderholdt; Swaroop Pophale; Manjunath Gorentla Venkata; Neena Imam

doi:10.1007/978-3-030-04918-8_1

OpenSHMEM sets and groups: An approach to worksharing and memory management

Ferrol Aderholdt, Swaroop Pophale, Manjunath Gorentla Venkata, Neena Imam

Programming Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Collective operations in the OpenSHMEM programming model are defined over an active set, which is a grouping of (PEs) based on a triple of information including the starting PE, a log₂ stride, and the size of the active set. In addition to the active set, collectives require Users to allocate and initialize synchronization (i.e., pSync) and scratchpad (i.e., pWrk) buffers for use by the collective operations. While active sets and the user-defined buffers were previously useful based on hardware and algorithmic considerations, future systems and applications require us to re-evaluate these concepts. In this paper, we propose Sets and Groups as abstractions to create persistent, flexible groupings of PEs (i.e., Sets) and couple these groups of PEs with memory spaces (i.e., Groups), which remove the allocation and initialization burden from the User. To evaluate Sets and Groups, we perform multiple micro-benchmarks to determine the overhead of these abstractions and demonstrate their utility by implementing a distributed APSP application, which we evaluate using multiple synthetic and real-world graphs.

Original language	English
Title of host publication	OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers
Editors	Swaroop Pophale, Neena Imam, Ferrol Aderholdt, Manjunath Gorentla Venkata
Publisher	Springer Verlag
Pages	3-21
Number of pages	19
ISBN (Print)	9783030049171
DOIs	https://doi.org/10.1007/978-3-030-04918-8_1
State	Published - 2019
Event	5th Workshop on OpenSHMEM and Related Technologies, 2018 - Baltimore, United States Duration: Aug 21 2018 → Aug 23 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11283 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	5th Workshop on OpenSHMEM and Related Technologies, 2018
Country/Territory	United States
City	Baltimore
Period	08/21/18 → 08/23/18

Funding

Acknowledgements. This research was supported by the United States Department of Defense (DoD) and Computational Research and Development Programs at Oak Ridge National Laboratory. This work was sponsored by the U.S. Department of Energy’s Office of Advanced Scientific Computing Research. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. F. Aderholdt—This work was sponsored by the U.S. Department of Energy’s Office of Advanced Scientific Computing Research. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy. gov/downloads/doe-public-access-plan). This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. F. Aderholdt—This work was sponsored by the U.S. Department of Energy’s Office of Advanced Scientific Computing Research. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscr‏ipt,‏ or ‏allow others‏ to do so, for Unite‏d ‏States Governm‏en‏t purposes‏. The Depart‏ment of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access ‏Plan‏ (ht‏tp‏://energy. gov/downloads/doe-public-access-plan). This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.This research was supported by the United States Department of Defense (DoD) and Computational Research and Development Programs at Oak Ridge National Laboratory. This work was sponsored by the U.S. Department of Energy’s Office of Advanced Scientific Computing Research. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Access to Document

10.1007/978-3-030-04918-8_1

Cite this

Aderholdt, F., Pophale, S., Gorentla Venkata, M., & Imam, N. (2019). OpenSHMEM sets and groups: An approach to worksharing and memory management. In S. Pophale, N. Imam, F. Aderholdt, & M. Gorentla Venkata (Eds.), OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers (pp. 3-21). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11283 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-04918-8_1

Aderholdt, Ferrol ; Pophale, Swaroop ; Gorentla Venkata, Manjunath et al. / OpenSHMEM sets and groups : An approach to worksharing and memory management. OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers. editor / Swaroop Pophale ; Neena Imam ; Ferrol Aderholdt ; Manjunath Gorentla Venkata. Springer Verlag, 2019. pp. 3-21 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{689d5290547c46198eeb491e1d38a591,

title = "OpenSHMEM sets and groups: An approach to worksharing and memory management",

abstract = "Collective operations in the OpenSHMEM programming model are defined over an active set, which is a grouping of (PEs) based on a triple of information including the starting PE, a log2 stride, and the size of the active set. In addition to the active set, collectives require Users to allocate and initialize synchronization (i.e., pSync) and scratchpad (i.e., pWrk) buffers for use by the collective operations. While active sets and the user-defined buffers were previously useful based on hardware and algorithmic considerations, future systems and applications require us to re-evaluate these concepts. In this paper, we propose Sets and Groups as abstractions to create persistent, flexible groupings of PEs (i.e., Sets) and couple these groups of PEs with memory spaces (i.e., Groups), which remove the allocation and initialization burden from the User. To evaluate Sets and Groups, we perform multiple micro-benchmarks to determine the overhead of these abstractions and demonstrate their utility by implementing a distributed APSP application, which we evaluate using multiple synthetic and real-world graphs.",

author = "Ferrol Aderholdt and Swaroop Pophale and {Gorentla Venkata}, Manjunath and Neena Imam",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 5th Workshop on OpenSHMEM and Related Technologies, 2018 ; Conference date: 21-08-2018 Through 23-08-2018",

year = "2019",

doi = "10.1007/978-3-030-04918-8_1",

language = "English",

isbn = "9783030049171",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "3--21",

editor = "Swaroop Pophale and Neena Imam and Ferrol Aderholdt and {Gorentla Venkata}, Manjunath",

booktitle = "OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers",

}

Aderholdt, F, Pophale, S, Gorentla Venkata, M & Imam, N 2019, OpenSHMEM sets and groups: An approach to worksharing and memory management. in S Pophale, N Imam, F Aderholdt & M Gorentla Venkata (eds), OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11283 LNCS, Springer Verlag, pp. 3-21, 5th Workshop on OpenSHMEM and Related Technologies, 2018, Baltimore, United States, 08/21/18. https://doi.org/10.1007/978-3-030-04918-8_1

OpenSHMEM sets and groups: An approach to worksharing and memory management. / Aderholdt, Ferrol; Pophale, Swaroop; Gorentla Venkata, Manjunath et al.
OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers. ed. / Swaroop Pophale; Neena Imam; Ferrol Aderholdt; Manjunath Gorentla Venkata. Springer Verlag, 2019. p. 3-21 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11283 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - OpenSHMEM sets and groups

T2 - 5th Workshop on OpenSHMEM and Related Technologies, 2018

AU - Aderholdt, Ferrol

AU - Pophale, Swaroop

AU - Gorentla Venkata, Manjunath

AU - Imam, Neena

PY - 2019

Y1 - 2019

N2 - Collective operations in the OpenSHMEM programming model are defined over an active set, which is a grouping of (PEs) based on a triple of information including the starting PE, a log2 stride, and the size of the active set. In addition to the active set, collectives require Users to allocate and initialize synchronization (i.e., pSync) and scratchpad (i.e., pWrk) buffers for use by the collective operations. While active sets and the user-defined buffers were previously useful based on hardware and algorithmic considerations, future systems and applications require us to re-evaluate these concepts. In this paper, we propose Sets and Groups as abstractions to create persistent, flexible groupings of PEs (i.e., Sets) and couple these groups of PEs with memory spaces (i.e., Groups), which remove the allocation and initialization burden from the User. To evaluate Sets and Groups, we perform multiple micro-benchmarks to determine the overhead of these abstractions and demonstrate their utility by implementing a distributed APSP application, which we evaluate using multiple synthetic and real-world graphs.

AB - Collective operations in the OpenSHMEM programming model are defined over an active set, which is a grouping of (PEs) based on a triple of information including the starting PE, a log2 stride, and the size of the active set. In addition to the active set, collectives require Users to allocate and initialize synchronization (i.e., pSync) and scratchpad (i.e., pWrk) buffers for use by the collective operations. While active sets and the user-defined buffers were previously useful based on hardware and algorithmic considerations, future systems and applications require us to re-evaluate these concepts. In this paper, we propose Sets and Groups as abstractions to create persistent, flexible groupings of PEs (i.e., Sets) and couple these groups of PEs with memory spaces (i.e., Groups), which remove the allocation and initialization burden from the User. To evaluate Sets and Groups, we perform multiple micro-benchmarks to determine the overhead of these abstractions and demonstrate their utility by implementing a distributed APSP application, which we evaluate using multiple synthetic and real-world graphs.

UR - http://www.scopus.com/inward/record.url?scp=85064314010&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-04918-8_1

DO - 10.1007/978-3-030-04918-8_1

M3 - Conference contribution

AN - SCOPUS:85064314010

SN - 9783030049171

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 3

EP - 21

BT - OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers

A2 - Pophale, Swaroop

A2 - Imam, Neena

A2 - Aderholdt, Ferrol

A2 - Gorentla Venkata, Manjunath

PB - Springer Verlag

Y2 - 21 August 2018 through 23 August 2018

ER -

Aderholdt F, Pophale S, Gorentla Venkata M, Imam N. OpenSHMEM sets and groups: An approach to worksharing and memory management. In Pophale S, Imam N, Aderholdt F, Gorentla Venkata M, editors, OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity - 5th Workshop, OpenSHMEM 2018, Revised Selected Papers. Springer Verlag. 2019. p. 3-21. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-04918-8_1

OpenSHMEM sets and groups: An approach to worksharing and memory management

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this