Improving the performance of the GMRES method using mixed-precision techniques

Neil Lindquist; Piotr Luszczek; Jack Dongarra

doi:10.1007/978-3-030-63393-6_4

Improving the performance of the GMRES method using mixed-precision techniques

Neil Lindquist, Piotr Luszczek, Jack Dongarra

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Scopus citations

Abstract

The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its computational kernels. To reduce the amount of data movement, and thus, to improve performance, we investigated the effect of using a mix of single and double precision while retaining double-precision accuracy. Previous efforts have explored reduced precision in the preconditioner, but the use of reduced precision in the solver itself has received limited attention. We found that GMRES only needs double precision in computing the residual and updating the approximate solution to achieve double-precision accuracy, although it must restart after each improvement of single-precision accuracy. This finding holds for the tested orthogonalization schemes: Modified Gram-Schmidt (MGS) and Classical Gram-Schmidt with Re-orthogonalization (CGSR). Furthermore, our mixed-precision GMRES, when restarted at least once, performed 19% and 24% faster on average than double-precision GMRES for MGS and CGSR, respectively. Our implementation uses generic programming techniques to ease the burden of coding implementations for different data types. Our use of the Kokkos library allowed us to exploit parallelism and optimize data management. Additionally, KokkosKernels was used when producing performance results. In conclusion, using a mix of single and double precision in GMRES can improve performance while retaining double-precision accuracy.

Original language	English
Title of host publication	Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers
Editors	Jeffrey Nichols, Arthur ‘Barney’ Maccabe, Suzanne Parete-Koon, Becky Verastegui, Oscar Hernandez, Theresa Ahearn
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	51-66
Number of pages	16
ISBN (Print)	9783030633929
DOIs	https://doi.org/10.1007/978-3-030-63393-6_4
State	Published - 2021
Event	17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020 - Virtual, Online Duration: Aug 26 2020 → Aug 28 2020

Publication series

Name	Communications in Computer and Information Science
Volume	1315 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020
City	Virtual, Online
Period	08/26/20 → 08/28/20

Funding

Acknowledgments. This material is based upon work supported by the University of Tennessee grant MSE E01-1315-038 as Interdisciplinary Seed funding and in part by UT Battelle subaward 4000123266. This material is also based upon work supported by the National Science Foundation under Grant No. 2004541.

Keywords

Kokkos
Krylov subspace methods
Linear algebra
Mixed precision

Access to Document

10.1007/978-3-030-63393-6_4

Cite this

Lindquist, N., Luszczek, P., & Dongarra, J. (2021). Improving the performance of the GMRES method using mixed-precision techniques. In J. Nichols, A. B. Maccabe, S. Parete-Koon, B. Verastegui, O. Hernandez, & T. Ahearn (Eds.), Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers (pp. 51-66). (Communications in Computer and Information Science; Vol. 1315 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-63393-6_4

Lindquist, Neil ; Luszczek, Piotr ; Dongarra, Jack. / Improving the performance of the GMRES method using mixed-precision techniques. Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers. editor / Jeffrey Nichols ; Arthur ‘Barney’ Maccabe ; Suzanne Parete-Koon ; Becky Verastegui ; Oscar Hernandez ; Theresa Ahearn. Springer Science and Business Media Deutschland GmbH, 2021. pp. 51-66 (Communications in Computer and Information Science).

@inproceedings{053814dce1b5471e9ea7ee68ae4930ba,

title = "Improving the performance of the GMRES method using mixed-precision techniques",

abstract = "The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its computational kernels. To reduce the amount of data movement, and thus, to improve performance, we investigated the effect of using a mix of single and double precision while retaining double-precision accuracy. Previous efforts have explored reduced precision in the preconditioner, but the use of reduced precision in the solver itself has received limited attention. We found that GMRES only needs double precision in computing the residual and updating the approximate solution to achieve double-precision accuracy, although it must restart after each improvement of single-precision accuracy. This finding holds for the tested orthogonalization schemes: Modified Gram-Schmidt (MGS) and Classical Gram-Schmidt with Re-orthogonalization (CGSR). Furthermore, our mixed-precision GMRES, when restarted at least once, performed 19% and 24% faster on average than double-precision GMRES for MGS and CGSR, respectively. Our implementation uses generic programming techniques to ease the burden of coding implementations for different data types. Our use of the Kokkos library allowed us to exploit parallelism and optimize data management. Additionally, KokkosKernels was used when producing performance results. In conclusion, using a mix of single and double precision in GMRES can improve performance while retaining double-precision accuracy.",

keywords = "Kokkos, Krylov subspace methods, Linear algebra, Mixed precision",

author = "Neil Lindquist and Piotr Luszczek and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020 ; Conference date: 26-08-2020 Through 28-08-2020",

year = "2021",

doi = "10.1007/978-3-030-63393-6_4",

language = "English",

isbn = "9783030633929",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "51--66",

editor = "Jeffrey Nichols and Maccabe, {Arthur {\textquoteleft}Barney{\textquoteright}} and Suzanne Parete-Koon and Becky Verastegui and Oscar Hernandez and Theresa Ahearn",

booktitle = "Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers",

}

Lindquist, N, Luszczek, P & Dongarra, J 2021, Improving the performance of the GMRES method using mixed-precision techniques. in J Nichols, AB Maccabe, S Parete-Koon, B Verastegui, O Hernandez & T Ahearn (eds), Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers. Communications in Computer and Information Science, vol. 1315 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 51-66, 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Virtual, Online, 08/26/20. https://doi.org/10.1007/978-3-030-63393-6_4

Improving the performance of the GMRES method using mixed-precision techniques. / Lindquist, Neil; Luszczek, Piotr; Dongarra, Jack.
Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers. ed. / Jeffrey Nichols; Arthur ‘Barney’ Maccabe; Suzanne Parete-Koon; Becky Verastegui; Oscar Hernandez; Theresa Ahearn. Springer Science and Business Media Deutschland GmbH, 2021. p. 51-66 (Communications in Computer and Information Science; Vol. 1315 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improving the performance of the GMRES method using mixed-precision techniques

AU - Lindquist, Neil

AU - Luszczek, Piotr

AU - Dongarra, Jack

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2021

Y1 - 2021

N2 - The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its computational kernels. To reduce the amount of data movement, and thus, to improve performance, we investigated the effect of using a mix of single and double precision while retaining double-precision accuracy. Previous efforts have explored reduced precision in the preconditioner, but the use of reduced precision in the solver itself has received limited attention. We found that GMRES only needs double precision in computing the residual and updating the approximate solution to achieve double-precision accuracy, although it must restart after each improvement of single-precision accuracy. This finding holds for the tested orthogonalization schemes: Modified Gram-Schmidt (MGS) and Classical Gram-Schmidt with Re-orthogonalization (CGSR). Furthermore, our mixed-precision GMRES, when restarted at least once, performed 19% and 24% faster on average than double-precision GMRES for MGS and CGSR, respectively. Our implementation uses generic programming techniques to ease the burden of coding implementations for different data types. Our use of the Kokkos library allowed us to exploit parallelism and optimize data management. Additionally, KokkosKernels was used when producing performance results. In conclusion, using a mix of single and double precision in GMRES can improve performance while retaining double-precision accuracy.

AB - The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its computational kernels. To reduce the amount of data movement, and thus, to improve performance, we investigated the effect of using a mix of single and double precision while retaining double-precision accuracy. Previous efforts have explored reduced precision in the preconditioner, but the use of reduced precision in the solver itself has received limited attention. We found that GMRES only needs double precision in computing the residual and updating the approximate solution to achieve double-precision accuracy, although it must restart after each improvement of single-precision accuracy. This finding holds for the tested orthogonalization schemes: Modified Gram-Schmidt (MGS) and Classical Gram-Schmidt with Re-orthogonalization (CGSR). Furthermore, our mixed-precision GMRES, when restarted at least once, performed 19% and 24% faster on average than double-precision GMRES for MGS and CGSR, respectively. Our implementation uses generic programming techniques to ease the burden of coding implementations for different data types. Our use of the Kokkos library allowed us to exploit parallelism and optimize data management. Additionally, KokkosKernels was used when producing performance results. In conclusion, using a mix of single and double precision in GMRES can improve performance while retaining double-precision accuracy.

KW - Kokkos

KW - Krylov subspace methods

KW - Linear algebra

KW - Mixed precision

UR - http://www.scopus.com/inward/record.url?scp=85107302954&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-63393-6_4

DO - 10.1007/978-3-030-63393-6_4

M3 - Conference contribution

AN - SCOPUS:85107302954

SN - 9783030633929

T3 - Communications in Computer and Information Science

SP - 51

EP - 66

BT - Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers

A2 - Nichols, Jeffrey

A2 - Maccabe, Arthur ‘Barney’

A2 - Parete-Koon, Suzanne

A2 - Verastegui, Becky

A2 - Hernandez, Oscar

A2 - Ahearn, Theresa

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020

Y2 - 26 August 2020 through 28 August 2020

ER -

Lindquist N, Luszczek P, Dongarra J. Improving the performance of the GMRES method using mixed-precision techniques. In Nichols J, Maccabe AB, Parete-Koon S, Verastegui B, Hernandez O, Ahearn T, editors, Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2021. p. 51-66. (Communications in Computer and Information Science). doi: 10.1007/978-3-030-63393-6_4

Improving the performance of the GMRES method using mixed-precision techniques

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this