Performance, design, and autotuning of batched GEMM for GPUs

Ahmad Abdelfattah; Azzam Haidar; Stanimire Tomov; Jack Dongarra

doi:10.1007/978-3-319-41321-1_2

Performance, design, and autotuning of batched GEMM for GPUs

Ahmad Abdelfattah, Azzam Haidar, Stanimire Tomov, Jack Dongarra

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

93 Scopus citations

Abstract

The general matrix-matrix multiplication (GEMM) is the most important numerical kernel in dense linear algebra, and is the key component for obtaining high performance in most LAPACK routines. As batched computations on relatively small problems continue to gain interest in many scientific applications, a need arises for a high performance GEMM kernel for batches of small matrices. Such a kernel should be well designed and tuned to handle small sizes, and to maintain high performance for realistic test cases found in the higher level LAPACK routines, and scientific computing applications in general. This paper presents a high performance batched GEMM kernel on Graphics Processing Units (GPUs). We address batched problems with both fixed and variable sizes, and show that specialized GEMM designs and a comprehensive autotuning process are needed to handle problems of small sizes. For most performance tests reported in this paper, the proposed kernels outperform state-of-the-art approaches using a K40c GPU.

Original language	English
Title of host publication	High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings
Editors	Jack Dongarra, Julian M. Kunkel, Pavan Balaji
Publisher	Springer Verlag
Pages	21-38
Number of pages	18
ISBN (Print)	9783319413204
DOIs	https://doi.org/10.1007/978-3-319-41321-1_2
State	Published - 2016
Event	31st International Conference on High Performance Computing, ISC High Performance 2016 - Frankfurt, Germany Duration: Jun 19 2016 → Jun 23 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9697
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	31st International Conference on High Performance Computing, ISC High Performance 2016
Country/Territory	Germany
City	Frankfurt
Period	06/19/16 → 06/23/16

Funding

This work is based upon work supported by the National Science Foundation under Grants No. ACI-1339822 and CSR 1514286, NVIDIA, the Department of Energy (LLNL subcontract under DOE contract DE-AC52-07NA27344), and in part by the Russian Scientific Foundation, Agreement N14-11-00190.

Keywords

Autotuning
Batched GEMM
GEMM
GPU computing
HPC

Access to Document

10.1007/978-3-319-41321-1_2

Cite this

Abdelfattah, A., Haidar, A., Tomov, S., & Dongarra, J. (2016). Performance, design, and autotuning of batched GEMM for GPUs. In J. Dongarra, J. M. Kunkel, & P. Balaji (Eds.), High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings (pp. 21-38). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9697). Springer Verlag. https://doi.org/10.1007/978-3-319-41321-1_2

Abdelfattah, Ahmad ; Haidar, Azzam ; Tomov, Stanimire et al. / Performance, design, and autotuning of batched GEMM for GPUs. High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings. editor / Jack Dongarra ; Julian M. Kunkel ; Pavan Balaji. Springer Verlag, 2016. pp. 21-38 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{5cc626cf0da944e0b06560906e8397e8,

title = "Performance, design, and autotuning of batched GEMM for GPUs",

abstract = "The general matrix-matrix multiplication (GEMM) is the most important numerical kernel in dense linear algebra, and is the key component for obtaining high performance in most LAPACK routines. As batched computations on relatively small problems continue to gain interest in many scientific applications, a need arises for a high performance GEMM kernel for batches of small matrices. Such a kernel should be well designed and tuned to handle small sizes, and to maintain high performance for realistic test cases found in the higher level LAPACK routines, and scientific computing applications in general. This paper presents a high performance batched GEMM kernel on Graphics Processing Units (GPUs). We address batched problems with both fixed and variable sizes, and show that specialized GEMM designs and a comprehensive autotuning process are needed to handle problems of small sizes. For most performance tests reported in this paper, the proposed kernels outperform state-of-the-art approaches using a K40c GPU.",

keywords = "Autotuning, Batched GEMM, GEMM, GPU computing, HPC",

author = "Ahmad Abdelfattah and Azzam Haidar and Stanimire Tomov and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 31st International Conference on High Performance Computing, ISC High Performance 2016 ; Conference date: 19-06-2016 Through 23-06-2016",

year = "2016",

doi = "10.1007/978-3-319-41321-1_2",

language = "English",

isbn = "9783319413204",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "21--38",

editor = "Jack Dongarra and Kunkel, {Julian M.} and Pavan Balaji",

booktitle = "High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings",

}

Abdelfattah, A, Haidar, A, Tomov, S & Dongarra, J 2016, Performance, design, and autotuning of batched GEMM for GPUs. in J Dongarra, JM Kunkel & P Balaji (eds), High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9697, Springer Verlag, pp. 21-38, 31st International Conference on High Performance Computing, ISC High Performance 2016, Frankfurt, Germany, 06/19/16. https://doi.org/10.1007/978-3-319-41321-1_2

Performance, design, and autotuning of batched GEMM for GPUs. / Abdelfattah, Ahmad; Haidar, Azzam; Tomov, Stanimire et al.
High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings. ed. / Jack Dongarra; Julian M. Kunkel; Pavan Balaji. Springer Verlag, 2016. p. 21-38 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9697).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Performance, design, and autotuning of batched GEMM for GPUs

AU - Abdelfattah, Ahmad

AU - Haidar, Azzam

AU - Tomov, Stanimire

AU - Dongarra, Jack

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - The general matrix-matrix multiplication (GEMM) is the most important numerical kernel in dense linear algebra, and is the key component for obtaining high performance in most LAPACK routines. As batched computations on relatively small problems continue to gain interest in many scientific applications, a need arises for a high performance GEMM kernel for batches of small matrices. Such a kernel should be well designed and tuned to handle small sizes, and to maintain high performance for realistic test cases found in the higher level LAPACK routines, and scientific computing applications in general. This paper presents a high performance batched GEMM kernel on Graphics Processing Units (GPUs). We address batched problems with both fixed and variable sizes, and show that specialized GEMM designs and a comprehensive autotuning process are needed to handle problems of small sizes. For most performance tests reported in this paper, the proposed kernels outperform state-of-the-art approaches using a K40c GPU.

AB - The general matrix-matrix multiplication (GEMM) is the most important numerical kernel in dense linear algebra, and is the key component for obtaining high performance in most LAPACK routines. As batched computations on relatively small problems continue to gain interest in many scientific applications, a need arises for a high performance GEMM kernel for batches of small matrices. Such a kernel should be well designed and tuned to handle small sizes, and to maintain high performance for realistic test cases found in the higher level LAPACK routines, and scientific computing applications in general. This paper presents a high performance batched GEMM kernel on Graphics Processing Units (GPUs). We address batched problems with both fixed and variable sizes, and show that specialized GEMM designs and a comprehensive autotuning process are needed to handle problems of small sizes. For most performance tests reported in this paper, the proposed kernels outperform state-of-the-art approaches using a K40c GPU.

KW - Autotuning

KW - Batched GEMM

KW - GEMM

KW - GPU computing

KW - HPC

UR - http://www.scopus.com/inward/record.url?scp=84977485193&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-41321-1_2

DO - 10.1007/978-3-319-41321-1_2

M3 - Conference contribution

AN - SCOPUS:84977485193

SN - 9783319413204

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 21

EP - 38

BT - High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings

A2 - Dongarra, Jack

A2 - Kunkel, Julian M.

A2 - Balaji, Pavan

PB - Springer Verlag

T2 - 31st International Conference on High Performance Computing, ISC High Performance 2016

Y2 - 19 June 2016 through 23 June 2016

ER -

Abdelfattah A, Haidar A, Tomov S, Dongarra J. Performance, design, and autotuning of batched GEMM for GPUs. In Dongarra J, Kunkel JM, Balaji P, editors, High Performance Computing - 31st International Conference, ISC High Performance 2016, Proceedings. Springer Verlag. 2016. p. 21-38. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-41321-1_2

Performance, design, and autotuning of batched GEMM for GPUs

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this