Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments

Azzam Haidar; Piotr Luszczek; Stanimire Tomov; Jack Dongarra

doi:10.1007/978-3-319-17353-5_3

Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments

Azzam Haidar, Piotr Luszczek, Stanimire Tomov, Jack Dongarra

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

We present an efficient and scalable programming model for the development of linear algebra in heterogeneous multi-coprocessor environments. The model incorporates some of the current best design and implementation practices for the heterogeneous acceleration of dense linear algebra (DLA). Examples are given as the basis for solving linear systems’ algorithms – the LU, QR, and Cholesky factorizations. To generate the extreme level of parallelism needed for the efficient use of coprocessors, algorithms of interest are redesigned and then split into well-chosen computational tasks. The tasks execution is scheduled over the computational components of a hybrid system of multi-core CPUs and coprocessors using a light-weight runtime system. The use of lightweight runtime systems keeps scheduling overhead low, while enabling the expression of parallelism through otherwise sequential code. This simplifies the development efforts and allows the exploration of the unique strengths of the various hardware components.

Original language	English
Title of host publication	High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers
Editors	Osni Marques, Michel Dayde, Kengo Nakajima
Publisher	Springer Verlag
Pages	31-42
Number of pages	12
ISBN (Print)	9783319173528
DOIs	https://doi.org/10.1007/978-3-319-17353-5_3
State	Published - 2015
Event	11th International Conference on High Performance Computing for Computational Science, VECPAR 2014 - Eugene, United States Duration: Jun 30 2014 → Jul 3 2014

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8969
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	11th International Conference on High Performance Computing for Computational Science, VECPAR 2014
Country/Territory	United States
City	Eugene
Period	06/30/14 → 07/3/14

Bibliographical note

Publisher Copyright:
© Springer International Publishing Switzerland 2015.

Funding

This research was partially supported by the National Science Foundation under Grants OCI-1032815, ACI-1339822, and Subcontract RA241-G1 on NSF Prime Grant OCI-0910735, DOE under Grants DE-SC0004983 and DE-SC0010042, and Intel. This research was supported in part by the National Science Foundation under Grants OCI-1032815, ACI-1339822, and Subcontract RA241-G1 on NSF Prime Grant OCI- 0910735, DOE under Grants DE-SC0004983 and DE-SC0010042, and Intel Corporation.

Funders	Funder number
National Science Foundation	RA241-G1
U.S. Department of Energy
National Science Foundation	OCI-1032815, OCI- 0910735, ACI-1339822
U.S. Department of Energy	DE-SC0004983, DE-SC0010042
Intel Corporation

Access to Document

10.1007/978-3-319-17353-5_3

Cite this

Haidar, A., Luszczek, P., Tomov, S., & Dongarra, J. (2015). Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments. In O. Marques, M. Dayde, & K. Nakajima (Eds.), High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers (pp. 31-42). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8969). Springer Verlag. https://doi.org/10.1007/978-3-319-17353-5_3

Haidar, Azzam ; Luszczek, Piotr ; Tomov, Stanimire et al. / Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments. High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers. editor / Osni Marques ; Michel Dayde ; Kengo Nakajima. Springer Verlag, 2015. pp. 31-42 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{dbf0ca53d31040c9b384af5922b89307,

title = "Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments",

abstract = "We present an efficient and scalable programming model for the development of linear algebra in heterogeneous multi-coprocessor environments. The model incorporates some of the current best design and implementation practices for the heterogeneous acceleration of dense linear algebra (DLA). Examples are given as the basis for solving linear systems{\textquoteright} algorithms – the LU, QR, and Cholesky factorizations. To generate the extreme level of parallelism needed for the efficient use of coprocessors, algorithms of interest are redesigned and then split into well-chosen computational tasks. The tasks execution is scheduled over the computational components of a hybrid system of multi-core CPUs and coprocessors using a light-weight runtime system. The use of lightweight runtime systems keeps scheduling overhead low, while enabling the expression of parallelism through otherwise sequential code. This simplifies the development efforts and allows the exploration of the unique strengths of the various hardware components.",

author = "Azzam Haidar and Piotr Luszczek and Stanimire Tomov and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 11th International Conference on High Performance Computing for Computational Science, VECPAR 2014 ; Conference date: 30-06-2014 Through 03-07-2014",

year = "2015",

doi = "10.1007/978-3-319-17353-5_3",

language = "English",

isbn = "9783319173528",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "31--42",

editor = "Osni Marques and Michel Dayde and Kengo Nakajima",

booktitle = "High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers",

}

Haidar, A, Luszczek, P, Tomov, S & Dongarra, J 2015, Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments. in O Marques, M Dayde & K Nakajima (eds), High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8969, Springer Verlag, pp. 31-42, 11th International Conference on High Performance Computing for Computational Science, VECPAR 2014, Eugene, United States, 06/30/14. https://doi.org/10.1007/978-3-319-17353-5_3

Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments. / Haidar, Azzam; Luszczek, Piotr; Tomov, Stanimire et al.
High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers. ed. / Osni Marques; Michel Dayde; Kengo Nakajima. Springer Verlag, 2015. p. 31-42 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8969).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments

AU - Haidar, Azzam

AU - Luszczek, Piotr

AU - Tomov, Stanimire

AU - Dongarra, Jack

PY - 2015

Y1 - 2015

N2 - We present an efficient and scalable programming model for the development of linear algebra in heterogeneous multi-coprocessor environments. The model incorporates some of the current best design and implementation practices for the heterogeneous acceleration of dense linear algebra (DLA). Examples are given as the basis for solving linear systems’ algorithms – the LU, QR, and Cholesky factorizations. To generate the extreme level of parallelism needed for the efficient use of coprocessors, algorithms of interest are redesigned and then split into well-chosen computational tasks. The tasks execution is scheduled over the computational components of a hybrid system of multi-core CPUs and coprocessors using a light-weight runtime system. The use of lightweight runtime systems keeps scheduling overhead low, while enabling the expression of parallelism through otherwise sequential code. This simplifies the development efforts and allows the exploration of the unique strengths of the various hardware components.

AB - We present an efficient and scalable programming model for the development of linear algebra in heterogeneous multi-coprocessor environments. The model incorporates some of the current best design and implementation practices for the heterogeneous acceleration of dense linear algebra (DLA). Examples are given as the basis for solving linear systems’ algorithms – the LU, QR, and Cholesky factorizations. To generate the extreme level of parallelism needed for the efficient use of coprocessors, algorithms of interest are redesigned and then split into well-chosen computational tasks. The tasks execution is scheduled over the computational components of a hybrid system of multi-core CPUs and coprocessors using a light-weight runtime system. The use of lightweight runtime systems keeps scheduling overhead low, while enabling the expression of parallelism through otherwise sequential code. This simplifies the development efforts and allows the exploration of the unique strengths of the various hardware components.

UR - http://www.scopus.com/inward/record.url?scp=84942593105&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-17353-5_3

DO - 10.1007/978-3-319-17353-5_3

M3 - Conference contribution

AN - SCOPUS:84942593105

SN - 9783319173528

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 31

EP - 42

BT - High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers

A2 - Marques, Osni

A2 - Dayde, Michel

A2 - Nakajima, Kengo

PB - Springer Verlag

T2 - 11th International Conference on High Performance Computing for Computational Science, VECPAR 2014

Y2 - 30 June 2014 through 3 July 2014

ER -

Haidar A, Luszczek P, Tomov S, Dongarra J. Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments. In Marques O, Dayde M, Nakajima K, editors, High Performance Computing for Computational Science - VECPAR 2014 - 11th International Conference, Revised Selected Papers. Springer Verlag. 2015. p. 31-42. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-17353-5_3

Heterogenous Acceleration for Linear Algebra in Multi-coprocessor environments

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this