Investigating the benefit of fp16-enabled mixed-precision solvers for symmetric positive definite matrices using gpus

Ahmad Abdelfattah, Stan Tomov, Jack Dongarra

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

9 Scopus citations

Abstract

Half-precision computation refers to performing floating-point operations in a 16-bit format. While half-precision has been driven largely by machine learning applications, recent algorithmic advances in numerical linear algebra have discovered beneficial use cases for half precision in accelerating the solution of linear systems of equations at higher precisions. In this paper, we present a high-performance, mixed-precision linear solver (Ax=b) for symmetric positive definite systems in double-precision using graphics processing units (GPUs). The solver is based on a mixed-precision Cholesky factorization that utilizes the high-performance tensor core units in CUDA-enabled GPUs. Since the Cholesky factors are affected by the low precision, an iterative refinement (IR) solver is required to recover the solution back to double-precision accuracy. Two different types of IR solvers are discussed on a wide range of test matrices. A preprocessing step is also developed, which scales and shifts the matrix, if necessary, in order to preserve its positive-definiteness in lower precisions. Our experiments on the V100 GPU show that performance speedups are up to 4.7× against a direct double-precision solver. However, matrix properties such as the condition number and the eigenvalue distribution can affect the convergence rate, which would consequently affect the overall performance.

Original languageEnglish
Title of host publicationComputational Science – ICCS 2020 - 20th International Conference, Proceedings
EditorsValeria V. Krzhizhanovskaya, Gábor Závodszky, Michael H. Lees, Peter M.A. Sloot, Peter M.A. Sloot, Peter M.A. Sloot, Jack J. Dongarra, Sérgio Brissos, João Teixeira
PublisherSpringer Science and Business Media Deutschland GmbH
Pages237-250
Number of pages14
ISBN (Print)9783030504168
DOIs
StatePublished - 2020
Event20th International Conference on Computational Science, ICCS 2020 - Amsterdam, Netherlands
Duration: Jun 3 2020Jun 5 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12138 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Computational Science, ICCS 2020
Country/TerritoryNetherlands
CityAmsterdam
Period06/3/2006/5/20

Keywords

  • GPU computing
  • Half-precision
  • Mixed-precision solvers

Fingerprint

Dive into the research topics of 'Investigating the benefit of fp16-enabled mixed-precision solvers for symmetric positive definite matrices using gpus'. Together they form a unique fingerprint.

Cite this