High-performance Matrix-Matrix multiplications of very small matrices

Ian Masliah; Ahmad Abdelfattah; A. Haidar; S. Tomov; Marc Baboulin; J. Falcou; J. Dongarra

doi:10.1007/978-3-319-43659-3_48

High-performance Matrix-Matrix multiplications of very small matrices

Ian Masliah, Ahmad Abdelfattah, A. Haidar, S. Tomov, Marc Baboulin, J. Falcou, J. Dongarra

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

40 Scopus citations

Abstract

The use of the general dense matrix-matrix multiplication (GEMM) is fundamental for obtaining high performance in many scientific computing applications. GEMMs for small matrices (of sizes less than 32) however, are not sufficiently optimized in existing libraries. In this paper we consider the case of many small GEMMs on either CPU or GPU architectures. This is a case that often occurs in applications like big data analytics, machine learning, high-order FEM, and others. The GEMMs are grouped together in a single batched routine. We present specialized for these cases algorithms and optimization techniques to obtain performance that is within 90% of the optimal. We show that these results outperform currently available state-of-the-art implementations and vendor-tuned math libraries.

Original language	English
Title of host publication	Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings
Editors	Pierre-François Dutot, Denis Trystram
Publisher	Springer Verlag
Pages	659-671
Number of pages	13
ISBN (Print)	9783319436586
DOIs	https://doi.org/10.1007/978-3-319-43659-3_48
State	Published - 2016
Externally published	Yes
Event	22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016 - Grenoble, France Duration: Aug 24 2016 → Aug 26 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9833 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016
Country/Territory	France
City	Grenoble
Period	08/24/16 → 08/26/16

Funding

This material is based in part upon work supported by the US NSF under Grants No. CSR 1514286 and ACI-1339822, NVIDIA, the Department of Energy, and in part by the Russian Scientific Foundation, Agreement N14-11-00190.

Funders	Funder number
National Science Foundation	CSR 1514286, ACI-1339822
U.S. Department of Energy
NVIDIA
Russian Science Foundation	N14-11-00190

Keywords

Autotuning
Batched GEMM
GEMM
HPC
Small matrices

Access to Document

10.1007/978-3-319-43659-3_48

Cite this

Masliah, I., Abdelfattah, A., Haidar, A., Tomov, S., Baboulin, M., Falcou, J., & Dongarra, J. (2016). High-performance Matrix-Matrix multiplications of very small matrices. In P.-F. Dutot, & D. Trystram (Eds.), Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings (pp. 659-671). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9833 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-43659-3_48

Masliah, Ian ; Abdelfattah, Ahmad ; Haidar, A. et al. / High-performance Matrix-Matrix multiplications of very small matrices. Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings. editor / Pierre-François Dutot ; Denis Trystram. Springer Verlag, 2016. pp. 659-671 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{12f30facc3864e139d8bb54b73e16a2c,

title = "High-performance Matrix-Matrix multiplications of very small matrices",

abstract = "The use of the general dense matrix-matrix multiplication (GEMM) is fundamental for obtaining high performance in many scientific computing applications. GEMMs for small matrices (of sizes less than 32) however, are not sufficiently optimized in existing libraries. In this paper we consider the case of many small GEMMs on either CPU or GPU architectures. This is a case that often occurs in applications like big data analytics, machine learning, high-order FEM, and others. The GEMMs are grouped together in a single batched routine. We present specialized for these cases algorithms and optimization techniques to obtain performance that is within 90% of the optimal. We show that these results outperform currently available state-of-the-art implementations and vendor-tuned math libraries.",

keywords = "Autotuning, Batched GEMM, GEMM, HPC, Small matrices",

author = "Ian Masliah and Ahmad Abdelfattah and A. Haidar and S. Tomov and Marc Baboulin and J. Falcou and J. Dongarra",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016 ; Conference date: 24-08-2016 Through 26-08-2016",

year = "2016",

doi = "10.1007/978-3-319-43659-3_48",

language = "English",

isbn = "9783319436586",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "659--671",

editor = "Pierre-Fran{\c c}ois Dutot and Denis Trystram",

booktitle = "Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings",

}

Masliah, I, Abdelfattah, A, Haidar, A, Tomov, S, Baboulin, M, Falcou, J & Dongarra, J 2016, High-performance Matrix-Matrix multiplications of very small matrices. in P-F Dutot & D Trystram (eds), Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9833 LNCS, Springer Verlag, pp. 659-671, 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Grenoble, France, 08/24/16. https://doi.org/10.1007/978-3-319-43659-3_48

High-performance Matrix-Matrix multiplications of very small matrices. / Masliah, Ian; Abdelfattah, Ahmad; Haidar, A. et al.
Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings. ed. / Pierre-François Dutot; Denis Trystram. Springer Verlag, 2016. p. 659-671 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9833 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - High-performance Matrix-Matrix multiplications of very small matrices

AU - Masliah, Ian

AU - Abdelfattah, Ahmad

AU - Haidar, A.

AU - Tomov, S.

AU - Baboulin, Marc

AU - Falcou, J.

AU - Dongarra, J.

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - The use of the general dense matrix-matrix multiplication (GEMM) is fundamental for obtaining high performance in many scientific computing applications. GEMMs for small matrices (of sizes less than 32) however, are not sufficiently optimized in existing libraries. In this paper we consider the case of many small GEMMs on either CPU or GPU architectures. This is a case that often occurs in applications like big data analytics, machine learning, high-order FEM, and others. The GEMMs are grouped together in a single batched routine. We present specialized for these cases algorithms and optimization techniques to obtain performance that is within 90% of the optimal. We show that these results outperform currently available state-of-the-art implementations and vendor-tuned math libraries.

AB - The use of the general dense matrix-matrix multiplication (GEMM) is fundamental for obtaining high performance in many scientific computing applications. GEMMs for small matrices (of sizes less than 32) however, are not sufficiently optimized in existing libraries. In this paper we consider the case of many small GEMMs on either CPU or GPU architectures. This is a case that often occurs in applications like big data analytics, machine learning, high-order FEM, and others. The GEMMs are grouped together in a single batched routine. We present specialized for these cases algorithms and optimization techniques to obtain performance that is within 90% of the optimal. We show that these results outperform currently available state-of-the-art implementations and vendor-tuned math libraries.

KW - Autotuning

KW - Batched GEMM

KW - GEMM

KW - HPC

KW - Small matrices

UR - http://www.scopus.com/inward/record.url?scp=84984780844&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-43659-3_48

DO - 10.1007/978-3-319-43659-3_48

M3 - Conference contribution

AN - SCOPUS:84984780844

SN - 9783319436586

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 659

EP - 671

BT - Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings

A2 - Dutot, Pierre-François

A2 - Trystram, Denis

PB - Springer Verlag

T2 - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016

Y2 - 24 August 2016 through 26 August 2016

ER -

Masliah I, Abdelfattah A, Haidar A, Tomov S, Baboulin M, Falcou J et al. High-performance Matrix-Matrix multiplications of very small matrices. In Dutot PF, Trystram D, editors, Parallel Processing - 22nd International Conference on Parallel and Distributed Computing, Euro-Par 2016, Proceedings. Springer Verlag. 2016. p. 659-671. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-43659-3_48

High-performance Matrix-Matrix multiplications of very small matrices

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this