A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

Ahmad Abdelfattah; Hartwig Anzt; Erik G. Boman; Erin Carson; Terry Cojean; Jack Dongarra; Alyson Fox; Mark Gates; Nicholas J. Higham; Xiaoye S. Li; Jennifer Loe; Piotr Luszczek; Srikara Pranesh; Siva Rajamanickam; Tobias Ribizel; Barry F. Smith; Kasia Swirydowicz; Stephen Thomas; Stanimire Tomov; Yaohung M. Tsai; Ulrike Meier Yang

doi:10.1177/10943420211003313

A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

Ahmad Abdelfattah, Hartwig Anzt, Erik G. Boman, Erin Carson, Terry Cojean, Jack Dongarra, Alyson Fox, Mark Gates, Nicholas J. Higham, Xiaoye S. Li, Jennifer Loe, Piotr Luszczek, Srikara Pranesh, Siva Rajamanickam, Tobias Ribizel, Barry F. Smith, Kasia Swirydowicz, Stephen Thomas, Stanimire Tomov, Yaohung M. TsaiUlrike Meier Yang

Computer Science and Mathematics Div

Research output: Contribution to journal › Article › peer-review

77 Scopus citations

Abstract

The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

Original language	English
Pages (from-to)	344-369
Number of pages	26
Journal	International Journal of High Performance Computing Applications
Volume	35
Issue number	4
DOIs	https://doi.org/10.1177/10943420211003313
State	Published - Jul 2021

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the US Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

Keywords

GPUs
Mixed-precision arithmetic
high-performance computing
linear algebra
numerical mathematics

Access to Document

10.1177/10943420211003313

Cite this

Abdelfattah, A., Anzt, H., Boman, E. G., Carson, E., Cojean, T., Dongarra, J., Fox, A., Gates, M., Higham, N. J., Li, X. S., Loe, J., Luszczek, P., Pranesh, S., Rajamanickam, S., Ribizel, T., Smith, B. F., Swirydowicz, K., Thomas, S., Tomov, S., ... Yang, U. M. (2021). A survey of numerical linear algebra methods utilizing mixed-precision arithmetic. International Journal of High Performance Computing Applications, 35(4), 344-369. https://doi.org/10.1177/10943420211003313

@article{7a28eae8b0be4d66a1667a1c940bc183,

title = "A survey of numerical linear algebra methods utilizing mixed-precision arithmetic",

abstract = "The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.",

keywords = "GPUs, Mixed-precision arithmetic, high-performance computing, linear algebra, numerical mathematics",

author = "Ahmad Abdelfattah and Hartwig Anzt and Boman, {Erik G.} and Erin Carson and Terry Cojean and Jack Dongarra and Alyson Fox and Mark Gates and Higham, {Nicholas J.} and Li, {Xiaoye S.} and Jennifer Loe and Piotr Luszczek and Srikara Pranesh and Siva Rajamanickam and Tobias Ribizel and Smith, {Barry F.} and Kasia Swirydowicz and Stephen Thomas and Stanimire Tomov and Tsai, {Yaohung M.} and Yang, {Ulrike Meier}",

note = "Publisher Copyright: {\textcopyright} The Author(s) 2021.",

year = "2021",

month = jul,

doi = "10.1177/10943420211003313",

language = "English",

volume = "35",

pages = "344--369",

journal = "International Journal of High Performance Computing Applications",

issn = "1094-3420",

publisher = "SAGE Publications",

number = "4",

}

Abdelfattah, A, Anzt, H, Boman, EG, Carson, E, Cojean, T, Dongarra, J, Fox, A, Gates, M, Higham, NJ, Li, XS, Loe, J, Luszczek, P, Pranesh, S, Rajamanickam, S, Ribizel, T, Smith, BF, Swirydowicz, K, Thomas, S, Tomov, S, Tsai, YM & Yang, UM 2021, 'A survey of numerical linear algebra methods utilizing mixed-precision arithmetic', International Journal of High Performance Computing Applications, vol. 35, no. 4, pp. 344-369. https://doi.org/10.1177/10943420211003313

TY - JOUR

T1 - A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

AU - Abdelfattah, Ahmad

AU - Anzt, Hartwig

AU - Boman, Erik G.

AU - Carson, Erin

AU - Cojean, Terry

AU - Dongarra, Jack

AU - Fox, Alyson

AU - Gates, Mark

AU - Higham, Nicholas J.

AU - Li, Xiaoye S.

AU - Loe, Jennifer

AU - Luszczek, Piotr

AU - Pranesh, Srikara

AU - Rajamanickam, Siva

AU - Ribizel, Tobias

AU - Smith, Barry F.

AU - Swirydowicz, Kasia

AU - Thomas, Stephen

AU - Tomov, Stanimire

AU - Tsai, Yaohung M.

AU - Yang, Ulrike Meier

PY - 2021/7

Y1 - 2021/7

N2 - The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

AB - The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

KW - GPUs

KW - Mixed-precision arithmetic

KW - high-performance computing

KW - linear algebra

KW - numerical mathematics

UR - http://www.scopus.com/inward/record.url?scp=85102726160&partnerID=8YFLogxK

U2 - 10.1177/10943420211003313

DO - 10.1177/10943420211003313

M3 - Article

AN - SCOPUS:85102726160

SN - 1094-3420

VL - 35

SP - 344

EP - 369

JO - International Journal of High Performance Computing Applications

JF - International Journal of High Performance Computing Applications

IS - 4

ER -

A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this