Weighted dynamic scheduling with many parallelism grains for offloading of numerical workloads to multiple varied accelerators

Azzam Haidar, Yulu Jia, Piotr Luszczek, Stanimire Tomov, Asim Yar Khan, Jack Dongarra

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

A wide variety of heterogeneous compute resources are available to modern computers, including multiple sockets containing multicore CPUs, one-or-more GPUS of varying power, and coprocessors such as the Intel Xeon Phi. The challenge faced by domain scientists is how to efficiently and productively use these varied resources. For example, in order to use GPUS effectively, the workload must have a greater degree of parallelism than a workload designed for a multicore-CPU. The domain scientist would have to design and schedule an application in multiple degrees of parallelism and task grain sizes in order to obtain efficient performance from the resources. We propose a productive programming model starting from serial code, which achieves parallelism and scalability by using a task-superscalar runtime environment to adapt the computation to the available resources. The adaptation is done at multiple points, including multi-level data partitioning, adaptive task grain sizes, and dynamic task scheduling. The effectiveness of this approach for utilizing multi-way heterogeneous hardware resources is demonstrated by implementing dense linear algebra applications.

Original languageEnglish
Title of host publicationProceedings of ScalA 2015
Subtitle of host publication6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450340113
DOIs
StatePublished - Nov 15 2015
Externally publishedYes
Event6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, ScalA 2015 - Austin, United States
Duration: Nov 15 2015Nov 20 2015

Publication series

NameProceedings of ScalA 2015: 6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, ScalA 2015
Country/TerritoryUnited States
CityAustin
Period11/15/1511/20/15

Funding

This material is based upon work supported by the National Science Foundation under Grant numbers ACI-1339822 and 1137097, and by the University of Tennessee through the Beacon project, the Department of Energy, and the NVIDIA and Intel Corporations. The results were obtained in part with the financial support of the Russian Scientific Fund, Agreement N14-11-00190.

FundersFunder number
Intel Corporations
Russian Scientific FundN14-11-00190
National Science Foundation1137097, ACI-1339822
U.S. Department of Energy
NVIDIA
University of Tennessee

    Keywords

    • Dataflow scheduling
    • Hardware accelerators
    • Multi-grain parallelism

    Fingerprint

    Dive into the research topics of 'Weighted dynamic scheduling with many parallelism grains for offloading of numerical workloads to multiple varied accelerators'. Together they form a unique fingerprint.

    Cite this