The design of fast and energy-efficient linear solvers: On the potential of half-precision arithmetic and iterative refinement techniques

Azzam Haidar, Ahmad Abdelfattah, Mawussi Zounon, Panruo Wu, Srikara Pranesh, Stanimire Tomov, Jack Dongarra

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

38 Scopus citations

Abstract

As parallel computers approach exascale, power efficiency in high-performance computing (HPC) systems is of increasing concern. Exploiting both the hardware features and algorithms is an effective solution to achieve power efficiency, and to address the energy constraints in modern and future HPC systems. In this work, we present a novel design and implementation of an energy-efficient solution for dense linear systems of equations, which are at the heart of large-scale HPC applications. The proposed energy-efficient linear system solvers are based on two main components: (1) iterative refinement techniques, and (2) reduced-precision computing features in modern accelerators and coprocessors. While most of the energy efficiency approaches aim to reduce the consumption with a minimal performance penalty, our method improves both the performance and the energy efficiency. Compared to highly-optimized linear system solvers, our kernels deliver the same accuracy solution up to 2× faster and reduce the energy consumption up to half on Intel Knights Landing (KNL) architectures. By efficiently using the Tensor Cores available in the NVIDIA V100 PCIe GPUs, the speedups can be up to 4×, with more than 80% reduction in the energy consumption.

Original languageEnglish
Title of host publicationComputational Science – ICCS 2018 - 18th International Conference, Proceedings
EditorsHaohuan Fu, Valeria V. Krzhizhanovskaya, Michael Harold Lees, Peter M. Sloot, Jack Dongarra, Yong Shi, Yingjie Tian
PublisherSpringer Verlag
Pages586-600
Number of pages15
ISBN (Print)9783319936970
DOIs
StatePublished - 2018
Event18th International Conference on Computational Science, ICCS 2018 - Wuxi, China
Duration: Jun 11 2018Jun 13 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10860 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference18th International Conference on Computational Science, ICCS 2018
Country/TerritoryChina
CityWuxi
Period06/11/1806/13/18

Funding

Acknowledgments. This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. The work was also partially supported by NVIDIA and NSF grant No. OAC-1740250.

Keywords

  • FP16
  • HPC
  • Mixed-precision
  • Solvers
  • Tensor cores

Fingerprint

Dive into the research topics of 'The design of fast and energy-efficient linear solvers: On the potential of half-precision arithmetic and iterative refinement techniques'. Together they form a unique fingerprint.

Cite this