Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs

Cade Brown; Ahmad Abdelfattah; Stanimire Tomov; Jack Dongarra

doi:10.1109/HPEC43674.2020.9286214

Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs

Cade Brown, Ahmad Abdelfattah, Stanimire Tomov, Jack Dongarra

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

19 Scopus citations

Abstract

Dense linear algebra (DLA) has historically been in the vanguard of software that must be adapted first to hardware changes. This is because DLA is both critical to the accuracy and performance of so many different types of applications, and because they have proved to be outstanding vehicles for finding and implementing solutions to the problems that novel architectures pose. Therefore, in this paper we investigate the portability of the MAGMA DLA library to the latest AMD GPUs. We use auto tools to convert the CUDA code in MAGMA to the Heterogeneous-Computing Interface for Portability (HIP) language. MAGMA provides LAPACK for GPUs and benchmarks for fundamental DLA routines ranging from BLAS to dense factorizations, linear systems and eigen-problem solvers. We port these routines to HIP and quantify currently achievable performance through the MAGMA benchmarks for the main workload algorithms on MI25 and MI50 AMD GPUs. Comparison with performance roofline models and theoretical expectations are used to identify current limitations and directions for future improvements.

Original language	English
Title of host publication	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728192192
DOIs	https://doi.org/10.1109/HPEC43674.2020.9286214
State	Published - Sep 22 2020
Externally published	Yes
Event	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020 - Virtual, Waltham, United States Duration: Sep 21 2020 → Sep 25 2020

Publication series

Name	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

Conference

Conference	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020
Country/Territory	United States
City	Virtual, Waltham
Period	09/21/20 → 09/25/20

Funding

This research was supported by AMD and the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC, a collaborative effort of two DOE organizations (the Office of Science and the National Nuclear Security Administration) responsible for the planning and preparation of a capable ex-ascale ecosystem, including software, applications, hardware, advanced system engineering, and early testbed platforms, in support of the nation’s exascale computing imperative.

Keywords

AMD GPUs
GPU Computing
HIP Runtime
HPC
Numerical Linear Algebra
Portability

Access to Document

10.1109/HPEC43674.2020.9286214

Cite this

Brown, C., Abdelfattah, A., Tomov, S., & Dongarra, J. (2020). Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs. In 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020 Article 9286214 (2020 IEEE High Performance Extreme Computing Conference, HPEC 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPEC43674.2020.9286214

@inproceedings{c1668414b61046248b19a8bc12700ba7,

title = "Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs",

abstract = "Dense linear algebra (DLA) has historically been in the vanguard of software that must be adapted first to hardware changes. This is because DLA is both critical to the accuracy and performance of so many different types of applications, and because they have proved to be outstanding vehicles for finding and implementing solutions to the problems that novel architectures pose. Therefore, in this paper we investigate the portability of the MAGMA DLA library to the latest AMD GPUs. We use auto tools to convert the CUDA code in MAGMA to the Heterogeneous-Computing Interface for Portability (HIP) language. MAGMA provides LAPACK for GPUs and benchmarks for fundamental DLA routines ranging from BLAS to dense factorizations, linear systems and eigen-problem solvers. We port these routines to HIP and quantify currently achievable performance through the MAGMA benchmarks for the main workload algorithms on MI25 and MI50 AMD GPUs. Comparison with performance roofline models and theoretical expectations are used to identify current limitations and directions for future improvements.",

keywords = "AMD GPUs, GPU Computing, HIP Runtime, HPC, Numerical Linear Algebra, Portability",

author = "Cade Brown and Ahmad Abdelfattah and Stanimire Tomov and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020 ; Conference date: 21-09-2020 Through 25-09-2020",

year = "2020",

month = sep,

day = "22",

doi = "10.1109/HPEC43674.2020.9286214",

language = "English",

series = "2020 IEEE High Performance Extreme Computing Conference, HPEC 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2020 IEEE High Performance Extreme Computing Conference, HPEC 2020",

}

Brown, C, Abdelfattah, A, Tomov, S & Dongarra, J 2020, Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs. in 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020., 9286214, 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020, Institute of Electrical and Electronics Engineers Inc., 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020, Virtual, Waltham, United States, 09/21/20. https://doi.org/10.1109/HPEC43674.2020.9286214

Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs. / Brown, Cade; Abdelfattah, Ahmad; Tomov, Stanimire et al.
2020 IEEE High Performance Extreme Computing Conference, HPEC 2020. Institute of Electrical and Electronics Engineers Inc., 2020. 9286214 (2020 IEEE High Performance Extreme Computing Conference, HPEC 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs

AU - Brown, Cade

AU - Abdelfattah, Ahmad

AU - Tomov, Stanimire

AU - Dongarra, Jack

PY - 2020/9/22

Y1 - 2020/9/22

N2 - Dense linear algebra (DLA) has historically been in the vanguard of software that must be adapted first to hardware changes. This is because DLA is both critical to the accuracy and performance of so many different types of applications, and because they have proved to be outstanding vehicles for finding and implementing solutions to the problems that novel architectures pose. Therefore, in this paper we investigate the portability of the MAGMA DLA library to the latest AMD GPUs. We use auto tools to convert the CUDA code in MAGMA to the Heterogeneous-Computing Interface for Portability (HIP) language. MAGMA provides LAPACK for GPUs and benchmarks for fundamental DLA routines ranging from BLAS to dense factorizations, linear systems and eigen-problem solvers. We port these routines to HIP and quantify currently achievable performance through the MAGMA benchmarks for the main workload algorithms on MI25 and MI50 AMD GPUs. Comparison with performance roofline models and theoretical expectations are used to identify current limitations and directions for future improvements.

AB - Dense linear algebra (DLA) has historically been in the vanguard of software that must be adapted first to hardware changes. This is because DLA is both critical to the accuracy and performance of so many different types of applications, and because they have proved to be outstanding vehicles for finding and implementing solutions to the problems that novel architectures pose. Therefore, in this paper we investigate the portability of the MAGMA DLA library to the latest AMD GPUs. We use auto tools to convert the CUDA code in MAGMA to the Heterogeneous-Computing Interface for Portability (HIP) language. MAGMA provides LAPACK for GPUs and benchmarks for fundamental DLA routines ranging from BLAS to dense factorizations, linear systems and eigen-problem solvers. We port these routines to HIP and quantify currently achievable performance through the MAGMA benchmarks for the main workload algorithms on MI25 and MI50 AMD GPUs. Comparison with performance roofline models and theoretical expectations are used to identify current limitations and directions for future improvements.

KW - AMD GPUs

KW - GPU Computing

KW - HIP Runtime

KW - HPC

KW - Numerical Linear Algebra

KW - Portability

UR - https://www.scopus.com/pages/publications/85099387939

U2 - 10.1109/HPEC43674.2020.9286214

DO - 10.1109/HPEC43674.2020.9286214

M3 - Conference contribution

AN - SCOPUS:85099387939

T3 - 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

BT - 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

Y2 - 21 September 2020 through 25 September 2020

ER -

Brown C, Abdelfattah A, Tomov S, Dongarra J. Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs. In 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020. Institute of Electrical and Electronics Engineers Inc. 2020. 9286214. (2020 IEEE High Performance Extreme Computing Conference, HPEC 2020). doi: 10.1109/HPEC43674.2020.9286214

Design, Optimization, and Benchmarking of Dense Linear Algebra Algorithms on AMD GPUs

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this