Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers

Azzam Haidar; Stanimire Tomov; Jack Dongarra; Nicholas J. Higham

doi:10.1109/SC.2018.00050

Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers

Azzam Haidar, Stanimire Tomov, Jack Dongarra, Nicholas J. Higham

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

155 Scopus citations

Abstract

Low-precision floating-point arithmetic is a powerful tool for accelerating scientific computing applications, especially those in artificial intelligence. Here, we present an investigation showing that other high-performance computing (HPC) applications can also harness this power. Specifically, we use the general HPC problem, Ax b, where A is a large dense matrix, and a double precision (FP64) solution is needed for accuracy. Our approach is based on mixed-precision (FP16-FP64) iterative refinement, and we generalize and extend prior advances into a framework, for which we develop architecture-specific algorithms and highly tuned implementations. These new methods show how using half-precision Tensor Cores (FP16-TC) for the arithmetic can provide up to 4× speedup. This is due to the performance boost that the FP16-TC provide as well as to the improved accuracy over the classical FP16 arithmetic that is obtained because the GEMM accumulation occurs in FP32 arithmetic.

Original language	English
Title of host publication	Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	603-613
Number of pages	11
ISBN (Electronic)	9781538683842
DOIs	https://doi.org/10.1109/SC.2018.00050
State	Published - Jul 2 2018
Event	2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018 - Dallas, United States Duration: Nov 11 2018 → Nov 16 2018

Publication series

Name	Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018

Conference

Conference	2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018
Country/Territory	United States
City	Dallas
Period	11/11/18 → 11/16/18

Funding

ACKNOWLEDGMENTS This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. The work was also partially supported by Nvidia and NSF grant No. OAC-1740250. N. J. Higham was supported by Engineering and Physical Sciences Research Council grant EP/P020720/1, The MathWorks, and the Royal Society.

Funders	Funder number
National Science Foundation	OAC-1740250
U.S. Department of Energy
National Nuclear Security Administration
Engineering and Physical Sciences Research Council	EP/P020720/1
Royal Society

Keywords

FP16 Arithmetic
GPU Computing
Half Precision
Iterative Refinement Computation
Linear Algebra
Mixed Precision Solvers

Access to Document

10.1109/SC.2018.00050

Cite this

Haidar, A., Tomov, S., Dongarra, J., & Higham, N. J. (2018). Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. In Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018 (pp. 603-613). Article 8665777 (Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SC.2018.00050

Haidar, Azzam ; Tomov, Stanimire ; Dongarra, Jack et al. / Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 603-613 (Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018).

@inproceedings{241eaab535b149a283a0fe3cf480dedc,

title = "Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers",

abstract = "Low-precision floating-point arithmetic is a powerful tool for accelerating scientific computing applications, especially those in artificial intelligence. Here, we present an investigation showing that other high-performance computing (HPC) applications can also harness this power. Specifically, we use the general HPC problem, Ax b, where A is a large dense matrix, and a double precision (FP64) solution is needed for accuracy. Our approach is based on mixed-precision (FP16-FP64) iterative refinement, and we generalize and extend prior advances into a framework, for which we develop architecture-specific algorithms and highly tuned implementations. These new methods show how using half-precision Tensor Cores (FP16-TC) for the arithmetic can provide up to 4× speedup. This is due to the performance boost that the FP16-TC provide as well as to the improved accuracy over the classical FP16 arithmetic that is obtained because the GEMM accumulation occurs in FP32 arithmetic.",

keywords = "FP16 Arithmetic, GPU Computing, Half Precision, Iterative Refinement Computation, Linear Algebra, Mixed Precision Solvers",

author = "Azzam Haidar and Stanimire Tomov and Jack Dongarra and Higham, {Nicholas J.}",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018 ; Conference date: 11-11-2018 Through 16-11-2018",

year = "2018",

month = jul,

day = "2",

doi = "10.1109/SC.2018.00050",

language = "English",

series = "Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "603--613",

booktitle = "Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018",

}

Haidar, A, Tomov, S, Dongarra, J & Higham, NJ 2018, Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. in Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018., 8665777, Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018, Institute of Electrical and Electronics Engineers Inc., pp. 603-613, 2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018, Dallas, United States, 11/11/18. https://doi.org/10.1109/SC.2018.00050

Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. / Haidar, Azzam; Tomov, Stanimire; Dongarra, Jack et al.
Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 603-613 8665777 (Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers

AU - Haidar, Azzam

AU - Tomov, Stanimire

AU - Dongarra, Jack

AU - Higham, Nicholas J.

PY - 2018/7/2

Y1 - 2018/7/2

N2 - Low-precision floating-point arithmetic is a powerful tool for accelerating scientific computing applications, especially those in artificial intelligence. Here, we present an investigation showing that other high-performance computing (HPC) applications can also harness this power. Specifically, we use the general HPC problem, Ax b, where A is a large dense matrix, and a double precision (FP64) solution is needed for accuracy. Our approach is based on mixed-precision (FP16-FP64) iterative refinement, and we generalize and extend prior advances into a framework, for which we develop architecture-specific algorithms and highly tuned implementations. These new methods show how using half-precision Tensor Cores (FP16-TC) for the arithmetic can provide up to 4× speedup. This is due to the performance boost that the FP16-TC provide as well as to the improved accuracy over the classical FP16 arithmetic that is obtained because the GEMM accumulation occurs in FP32 arithmetic.

AB - Low-precision floating-point arithmetic is a powerful tool for accelerating scientific computing applications, especially those in artificial intelligence. Here, we present an investigation showing that other high-performance computing (HPC) applications can also harness this power. Specifically, we use the general HPC problem, Ax b, where A is a large dense matrix, and a double precision (FP64) solution is needed for accuracy. Our approach is based on mixed-precision (FP16-FP64) iterative refinement, and we generalize and extend prior advances into a framework, for which we develop architecture-specific algorithms and highly tuned implementations. These new methods show how using half-precision Tensor Cores (FP16-TC) for the arithmetic can provide up to 4× speedup. This is due to the performance boost that the FP16-TC provide as well as to the improved accuracy over the classical FP16 arithmetic that is obtained because the GEMM accumulation occurs in FP32 arithmetic.

KW - FP16 Arithmetic

KW - GPU Computing

KW - Half Precision

KW - Iterative Refinement Computation

KW - Linear Algebra

KW - Mixed Precision Solvers

UR - http://www.scopus.com/inward/record.url?scp=85062956114&partnerID=8YFLogxK

U2 - 10.1109/SC.2018.00050

DO - 10.1109/SC.2018.00050

M3 - Conference contribution

AN - SCOPUS:85062956114

T3 - Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018

SP - 603

EP - 613

BT - Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018

Y2 - 11 November 2018 through 16 November 2018

ER -

Haidar A, Tomov S, Dongarra J, Higham NJ. Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. In Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 603-613. 8665777. (Proceedings - International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018). doi: 10.1109/SC.2018.00050

Harnessing GPU Tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this