Deep Gaussian process with multitask and transfer learning for performance optimization

Wissam M. Sid-Lakhdar; Mohsen Aznaveh; Piotr Luszczek; Jack Dongarra

doi:10.1109/HPEC55821.2022.9926396

Deep Gaussian process with multitask and transfer learning for performance optimization

Wissam M. Sid-Lakhdar, Mohsen Aznaveh, Piotr Luszczek, Jack Dongarra

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We combine Deep Gaussian Processes with multitask and transfer learning for the performance modeling and optimization of HPC applications. Deep Gaussian processes merge the uncertainty quantification advantage of Gaussian Processes with the predictive power of deep learning. Multitask and transfer learning allow for improved learning efficiency when several similar tasks are to be learned simultaneously and when previous learned models are sought to help in the learning of new tasks, respectively. A comparison with state-of-the-art autotuners shows the advantage of our approach on two application problems.

Original language	English
Title of host publication	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665497862
DOIs	https://doi.org/10.1109/HPEC55821.2022.9926396
State	Published - 2022
Externally published	Yes
Event	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 - Virtual, Online, United States Duration: Sep 19 2022 → Sep 23 2022

Publication series

Name	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

Conference

Conference	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022
Country/Territory	United States
City	Virtual, Online
Period	09/19/22 → 09/23/22

Funding

This work was supported by the U.S. Department of Energy, Office of Science, ASCR under Award Number DE-SC0021419 the National Science Foundation under OAC grant No. 2004541.

Funders	Funder number
National Science Foundation
U.S. Department of Energy
Ohio Arts Council	2004541
Office of Science
Advanced Scientific Computing Research	DE-SC0021419

Access to Document

10.1109/HPEC55821.2022.9926396

Cite this

Sid-Lakhdar, W. M., Aznaveh, M., Luszczek, P., & Dongarra, J. (2022). Deep Gaussian process with multitask and transfer learning for performance optimization. In 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPEC55821.2022.9926396

@inproceedings{f3304b17c2404cfcb0aaa65e74d8495d,

title = "Deep Gaussian process with multitask and transfer learning for performance optimization",

abstract = "We combine Deep Gaussian Processes with multitask and transfer learning for the performance modeling and optimization of HPC applications. Deep Gaussian processes merge the uncertainty quantification advantage of Gaussian Processes with the predictive power of deep learning. Multitask and transfer learning allow for improved learning efficiency when several similar tasks are to be learned simultaneously and when previous learned models are sought to help in the learning of new tasks, respectively. A comparison with state-of-the-art autotuners shows the advantage of our approach on two application problems.",

author = "Sid-Lakhdar, {Wissam M.} and Mohsen Aznaveh and Piotr Luszczek and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 ; Conference date: 19-09-2022 Through 23-09-2022",

year = "2022",

doi = "10.1109/HPEC55821.2022.9926396",

language = "English",

series = "2022 IEEE High Performance Extreme Computing Conference, HPEC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2022 IEEE High Performance Extreme Computing Conference, HPEC 2022",

}

Sid-Lakhdar, WM, Aznaveh, M, Luszczek, P & Dongarra, J 2022, Deep Gaussian process with multitask and transfer learning for performance optimization. in 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, Institute of Electrical and Electronics Engineers Inc., 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, Virtual, Online, United States, 09/19/22. https://doi.org/10.1109/HPEC55821.2022.9926396

Deep Gaussian process with multitask and transfer learning for performance optimization. / Sid-Lakhdar, Wissam M.; Aznaveh, Mohsen; Luszczek, Piotr et al.
2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Deep Gaussian process with multitask and transfer learning for performance optimization

AU - Sid-Lakhdar, Wissam M.

AU - Aznaveh, Mohsen

AU - Luszczek, Piotr

AU - Dongarra, Jack

PY - 2022

Y1 - 2022

N2 - We combine Deep Gaussian Processes with multitask and transfer learning for the performance modeling and optimization of HPC applications. Deep Gaussian processes merge the uncertainty quantification advantage of Gaussian Processes with the predictive power of deep learning. Multitask and transfer learning allow for improved learning efficiency when several similar tasks are to be learned simultaneously and when previous learned models are sought to help in the learning of new tasks, respectively. A comparison with state-of-the-art autotuners shows the advantage of our approach on two application problems.

AB - We combine Deep Gaussian Processes with multitask and transfer learning for the performance modeling and optimization of HPC applications. Deep Gaussian processes merge the uncertainty quantification advantage of Gaussian Processes with the predictive power of deep learning. Multitask and transfer learning allow for improved learning efficiency when several similar tasks are to be learned simultaneously and when previous learned models are sought to help in the learning of new tasks, respectively. A comparison with state-of-the-art autotuners shows the advantage of our approach on two application problems.

UR - http://www.scopus.com/inward/record.url?scp=85142272791&partnerID=8YFLogxK

U2 - 10.1109/HPEC55821.2022.9926396

DO - 10.1109/HPEC55821.2022.9926396

M3 - Conference contribution

AN - SCOPUS:85142272791

T3 - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

BT - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

Y2 - 19 September 2022 through 23 September 2022

ER -

Sid-Lakhdar WM, Aznaveh M, Luszczek P, Dongarra J. Deep Gaussian process with multitask and transfer learning for performance optimization. In 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. Institute of Electrical and Electronics Engineers Inc. 2022. (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022). doi: 10.1109/HPEC55821.2022.9926396

Deep Gaussian process with multitask and transfer learning for performance optimization

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this