MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing

Daniel Nichols; Nathalie Sofia Tomov; Frank Betancourt; Stanimire Tomov; Kwai Wong; Jack Dongarra

doi:10.1007/978-3-030-34356-9_37

MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing

Daniel Nichols, Nathalie Sofia Tomov, Frank Betancourt, Stanimire Tomov, Kwai Wong, Jack Dongarra

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.

Original language	English
Title of host publication	High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers
Editors	Michèle Weiland, Guido Juckeland, Sadaf Alam, Heike Jagode
Publisher	Springer
Pages	490-503
Number of pages	14
ISBN (Print)	9783030343552
DOIs	https://doi.org/10.1007/978-3-030-34356-9_37
State	Published - 2019
Event	34th International Conference on High Performance Computing, ISC High Performance 2019 - Frankfurt, Germany Duration: Jun 16 2019 → Jun 20 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11887 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	34th International Conference on High Performance Computing, ISC High Performance 2019
Country/Territory	Germany
City	Frankfurt
Period	06/16/19 → 06/20/19

Funding

This work was conducted at the Joint Institute for Computational Sciences (JICS) and the Innovative Computing Laboratory (ICL). This work is sponsored by the National Science Foundation (NSF), through NSF REU Award #1659502, with additional Support from the University of Tennessee, Knoxville (UTK), the National Institute for Computational Sciences (NICS), and NSF Awards #1740250 and #1709069. This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by NSF grant #ACI-1548562. Computational Resources are available through a XSEDE education allocation awards TG-ASC170031 and TG-ASC190013. In addition, the computing work was also performed on technical workstations donated by the BP High Performance Computing Team, as well as on GPUs donated by NVIDIA.

Keywords

Data-driven scientific computing
High-performance DNN
Machine learning

Access to Document

10.1007/978-3-030-34356-9_37

Cite this

Nichols, D., Tomov, N. S., Betancourt, F., Tomov, S., Wong, K., & Dongarra, J. (2019). MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing. In M. Weiland, G. Juckeland, S. Alam, & H. Jagode (Eds.), High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers (pp. 490-503). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11887 LNCS). Springer. https://doi.org/10.1007/978-3-030-34356-9_37

Nichols, Daniel ; Tomov, Nathalie Sofia ; Betancourt, Frank et al. / MagmaDNN : Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing. High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. editor / Michèle Weiland ; Guido Juckeland ; Sadaf Alam ; Heike Jagode. Springer, 2019. pp. 490-503 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{98b35e8c2fd841eb84947917585ebc62,

title = "MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing",

abstract = "In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.",

keywords = "Data-driven scientific computing, High-performance DNN, Machine learning",

author = "Daniel Nichols and Tomov, {Nathalie Sofia} and Frank Betancourt and Stanimire Tomov and Kwai Wong and Jack Dongarra",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 34th International Conference on High Performance Computing, ISC High Performance 2019 ; Conference date: 16-06-2019 Through 20-06-2019",

year = "2019",

doi = "10.1007/978-3-030-34356-9_37",

language = "English",

isbn = "9783030343552",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "490--503",

editor = "Mich{\`e}le Weiland and Guido Juckeland and Sadaf Alam and Heike Jagode",

booktitle = "High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers",

}

Nichols, D, Tomov, NS, Betancourt, F, Tomov, S, Wong, K & Dongarra, J 2019, MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing. in M Weiland, G Juckeland, S Alam & H Jagode (eds), High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11887 LNCS, Springer, pp. 490-503, 34th International Conference on High Performance Computing, ISC High Performance 2019, Frankfurt, Germany, 06/16/19. https://doi.org/10.1007/978-3-030-34356-9_37

MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing. / Nichols, Daniel; Tomov, Nathalie Sofia; Betancourt, Frank et al.
High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. ed. / Michèle Weiland; Guido Juckeland; Sadaf Alam; Heike Jagode. Springer, 2019. p. 490-503 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11887 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - MagmaDNN

T2 - 34th International Conference on High Performance Computing, ISC High Performance 2019

AU - Nichols, Daniel

AU - Tomov, Nathalie Sofia

AU - Betancourt, Frank

AU - Tomov, Stanimire

AU - Wong, Kwai

AU - Dongarra, Jack

PY - 2019

Y1 - 2019

N2 - In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.

AB - In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.

KW - Data-driven scientific computing

KW - High-performance DNN

KW - Machine learning

UR - http://www.scopus.com/inward/record.url?scp=85076839916&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-34356-9_37

DO - 10.1007/978-3-030-34356-9_37

M3 - Conference contribution

AN - SCOPUS:85076839916

SN - 9783030343552

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 490

EP - 503

BT - High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers

A2 - Weiland, Michèle

A2 - Juckeland, Guido

A2 - Alam, Sadaf

A2 - Jagode, Heike

PB - Springer

Y2 - 16 June 2019 through 20 June 2019

ER -

Nichols D, Tomov NS, Betancourt F, Tomov S, Wong K, Dongarra J. MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing. In Weiland M, Juckeland G, Alam S, Jagode H, editors, High Performance Computing - ISC High Performance 2019 International Workshops, Revised Selected Papers. Springer. 2019. p. 490-503. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-34356-9_37

MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this