The Design and Performance of Batched BLAS on Modern High-Performance Computing Systems

Jack Dongarra; Sven Hammarling; Nicholas J. Higham; Samuel D. Relton; Pedro Valero-Lara; Mawussi Zounon

doi:10.1016/j.procs.2017.05.138

The Design and Performance of Batched BLAS on Modern High-Performance Computing Systems

Jack Dongarra, Sven Hammarling, Nicholas J. Higham, Samuel D. Relton, Pedro Valero-Lara, Mawussi Zounon

Computer Science and Mathematics Div

Research output: Contribution to journal › Conference article › peer-review

54 Scopus citations

Abstract

A current trend in high-performance computing is to decompose a large linear algebra problem into batches containing thousands of smaller problems, that can be solved independently, before collating the results. To standardize the interface to these routines, the community is developing an extension to the BLAS standard (the batched BLAS), enabling users to perform thousands of small BLAS operations in parallel whilst making efficient use of their hardware. We discuss the benefits and drawbacks of the current batched BLAS proposals and perform a number of experiments, focusing on a general matrix-matrix multiplication (GEMM), to explore their affect on the performance. In particular we analyze the effect of novel data layouts which, for example, interleave the matrices in memory to aid vectorization and prefetching of data. Utilizing these modifications our code outperforms both MKL¹ CuBLAS² by up to 6 times on the self-hosted Intel KNL (codenamed Knights Landing) and Kepler GPU architectures, for large numbers of double precision GEMM operations using matrices of size 2 × 2 to 20 × 20.

Original language	English
Pages (from-to)	495-504
Number of pages	10
Journal	Procedia Computer Science
Volume	108
DOIs	https://doi.org/10.1016/j.procs.2017.05.138
State	Published - 2017
Event	International Conference on Computational Science ICCS 2017 - Zurich, Switzerland Duration: Jun 12 2017 → Jun 14 2017

Funding

The authors would like to thank The University of Tennessee for the use of their computational resources. This research was funded in part from the European Unions Horizon 2020 research and innovation programme under the NLAFET grant agreement No. 671633.

Funders	Funder number
University of Tennessee
Horizon 2020 Framework Programme	671633

Keywords

BLAS
Batched BLAS
High-performance computing
Memory management
Parallel processing
Scientific computing

Access to Document

10.1016/j.procs.2017.05.138

Cite this

@article{de23457a3afb41a6acdba47e8ae09358,

title = "The Design and Performance of Batched BLAS on Modern High-Performance Computing Systems",

abstract = "A current trend in high-performance computing is to decompose a large linear algebra problem into batches containing thousands of smaller problems, that can be solved independently, before collating the results. To standardize the interface to these routines, the community is developing an extension to the BLAS standard (the batched BLAS), enabling users to perform thousands of small BLAS operations in parallel whilst making efficient use of their hardware. We discuss the benefits and drawbacks of the current batched BLAS proposals and perform a number of experiments, focusing on a general matrix-matrix multiplication (GEMM), to explore their affect on the performance. In particular we analyze the effect of novel data layouts which, for example, interleave the matrices in memory to aid vectorization and prefetching of data. Utilizing these modifications our code outperforms both MKL1 CuBLAS2 by up to 6 times on the self-hosted Intel KNL (codenamed Knights Landing) and Kepler GPU architectures, for large numbers of double precision GEMM operations using matrices of size 2 × 2 to 20 × 20.",

keywords = "BLAS, Batched BLAS, High-performance computing, Memory management, Parallel processing, Scientific computing",

author = "Jack Dongarra and Sven Hammarling and Higham, {Nicholas J.} and Relton, {Samuel D.} and Pedro Valero-Lara and Mawussi Zounon",

note = "Publisher Copyright: {\textcopyright} 2017 The Authors. Published by Elsevier B.V.; International Conference on Computational Science ICCS 2017 ; Conference date: 12-06-2017 Through 14-06-2017",

year = "2017",

doi = "10.1016/j.procs.2017.05.138",

language = "English",

volume = "108",

pages = "495--504",

journal = "Procedia Computer Science",

issn = "1877-0509",

publisher = "Elsevier",

}

TY - JOUR

T1 - The Design and Performance of Batched BLAS on Modern High-Performance Computing Systems

AU - Dongarra, Jack

AU - Hammarling, Sven

AU - Higham, Nicholas J.

AU - Relton, Samuel D.

AU - Valero-Lara, Pedro

AU - Zounon, Mawussi

PY - 2017

Y1 - 2017

N2 - A current trend in high-performance computing is to decompose a large linear algebra problem into batches containing thousands of smaller problems, that can be solved independently, before collating the results. To standardize the interface to these routines, the community is developing an extension to the BLAS standard (the batched BLAS), enabling users to perform thousands of small BLAS operations in parallel whilst making efficient use of their hardware. We discuss the benefits and drawbacks of the current batched BLAS proposals and perform a number of experiments, focusing on a general matrix-matrix multiplication (GEMM), to explore their affect on the performance. In particular we analyze the effect of novel data layouts which, for example, interleave the matrices in memory to aid vectorization and prefetching of data. Utilizing these modifications our code outperforms both MKL1 CuBLAS2 by up to 6 times on the self-hosted Intel KNL (codenamed Knights Landing) and Kepler GPU architectures, for large numbers of double precision GEMM operations using matrices of size 2 × 2 to 20 × 20.

AB - A current trend in high-performance computing is to decompose a large linear algebra problem into batches containing thousands of smaller problems, that can be solved independently, before collating the results. To standardize the interface to these routines, the community is developing an extension to the BLAS standard (the batched BLAS), enabling users to perform thousands of small BLAS operations in parallel whilst making efficient use of their hardware. We discuss the benefits and drawbacks of the current batched BLAS proposals and perform a number of experiments, focusing on a general matrix-matrix multiplication (GEMM), to explore their affect on the performance. In particular we analyze the effect of novel data layouts which, for example, interleave the matrices in memory to aid vectorization and prefetching of data. Utilizing these modifications our code outperforms both MKL1 CuBLAS2 by up to 6 times on the self-hosted Intel KNL (codenamed Knights Landing) and Kepler GPU architectures, for large numbers of double precision GEMM operations using matrices of size 2 × 2 to 20 × 20.

KW - BLAS

KW - Batched BLAS

KW - High-performance computing

KW - Memory management

KW - Parallel processing

KW - Scientific computing

UR - http://www.scopus.com/inward/record.url?scp=85027356931&partnerID=8YFLogxK

U2 - 10.1016/j.procs.2017.05.138

DO - 10.1016/j.procs.2017.05.138

M3 - Conference article

AN - SCOPUS:85027356931

SN - 1877-0509

VL - 108

SP - 495

EP - 504

JO - Procedia Computer Science

JF - Procedia Computer Science

T2 - International Conference on Computational Science ICCS 2017

Y2 - 12 June 2017 through 14 June 2017

ER -

The Design and Performance of Batched BLAS on Modern High-Performance Computing Systems

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this