A scalable approach to solving dense linear algebra problems on hybrid CPU-GPU systems

Fengguang Song, Jack Dongarra

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

Aiming to fully exploit the computing power of all CPUs and all graphics processing units (GPUs) on hybrid CPU-GPU systems to solve dense linear algebra problems, we design a class of heterogeneous tile algorithms to maximize the degree of parallelism, to minimize the communication volume, and to accommodate the heterogeneity between CPUs and GPUs. The new heterogeneous tile algorithms are executed upon our decentralized dynamic scheduling runtime system, which schedules a task graph dynamically and transfers data between compute nodes automatically. The runtime system uses a new distributed task assignment protocol to solve data dependencies between tasks without any coordination between processing units. By overlapping computation and communication through dynamic scheduling, we are able to attain scalable performance for the double-precision Cholesky factorization and QR factorization. Our approach demonstrates a performance comparable to Intel MKL on shared-memory multicore systems and better performance than both vendor (e.g., Intel MKL) and open source libraries (e.g., StarPU) in the following three environments: heterogeneous clusters with GPUs, conventional clusters without GPUs, and shared-memory systems with multiple GPUs.

Original languageEnglish
Pages (from-to)3702-3723
Number of pages22
JournalConcurrency and Computation: Practice and Experience
Volume27
Issue number14
DOIs
StatePublished - Sep 25 2015

Keywords

  • dense linear algebra
  • distributed dataflow scheduling
  • heterogeneous HPC systems
  • runtime systems

Fingerprint

Dive into the research topics of 'A scalable approach to solving dense linear algebra problems on hybrid CPU-GPU systems'. Together they form a unique fingerprint.

Cite this