Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling

Piotr Luszczek; Jack Dongarra

doi:10.1007/978-3-642-31464-3_74

Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling

Piotr Luszczek, Jack Dongarra

Computer Science and Mathematics Div

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

We present a modeling framework to accurately predict time to run dense linear algebra calculation. We report the framework's accuracy in a number of varied computational environments such as shared memory multicore systems, clusters, and large supercomputing installations with tens of thousands of cores. We also test the accuracy for various algorithms, each of which having a different scaling properties and tolerance to low-bandwidth/high-latency interconnects. The predictive accuracy is very good and on the order of measurement accuracy which makes the method suitable for both dedicated and non-dedicated environments. We also present a practical application of our model to reduce the time required to tune and optimize large parallel runs whose time is dominated by linear algebra computations. We show practical examples of how to apply the methodology to avoid common pitfalls and reduce the influence of measurement errors and the inherent performance variability.

Original language	English
Title of host publication	Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers
Pages	730-739
Number of pages	10
Edition	PART 1
DOIs	https://doi.org/10.1007/978-3-642-31464-3_74
State	Published - 2012
Event	9th International Conference on Parallel Processing and Applied Mathematics, PPAM 2011 - Torun, Poland Duration: Sep 11 2011 → Sep 14 2011

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 1
Volume	7203 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Conference on Parallel Processing and Applied Mathematics, PPAM 2011
Country/Territory	Poland
City	Torun
Period	09/11/11 → 09/14/11

Funding

This research was supported by DARPA through ORNL subcontract 4000075916 as well as NSF through award number 1038814. We would like to also thank Patrick Worley from ORNL for facilitating the large scale runs on Jaguar’s Cray XT4 partition.

Keywords

Linear systems
modeling techniques
parallel algorithms

Access to Document

10.1007/978-3-642-31464-3_74

Cite this

Luszczek, P., & Dongarra, J. (2012). Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling. In Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers (PART 1 ed., pp. 730-739). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7203 LNCS, No. PART 1). https://doi.org/10.1007/978-3-642-31464-3_74

Luszczek, Piotr ; Dongarra, Jack. / Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling. Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers. PART 1. ed. 2012. pp. 730-739 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).

@inproceedings{72267c2877194b88964303e092473983,

title = "Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling",

abstract = "We present a modeling framework to accurately predict time to run dense linear algebra calculation. We report the framework's accuracy in a number of varied computational environments such as shared memory multicore systems, clusters, and large supercomputing installations with tens of thousands of cores. We also test the accuracy for various algorithms, each of which having a different scaling properties and tolerance to low-bandwidth/high-latency interconnects. The predictive accuracy is very good and on the order of measurement accuracy which makes the method suitable for both dedicated and non-dedicated environments. We also present a practical application of our model to reduce the time required to tune and optimize large parallel runs whose time is dominated by linear algebra computations. We show practical examples of how to apply the methodology to avoid common pitfalls and reduce the influence of measurement errors and the inherent performance variability.",

keywords = "Linear systems, modeling techniques, parallel algorithms",

author = "Piotr Luszczek and Jack Dongarra",

year = "2012",

doi = "10.1007/978-3-642-31464-3_74",

language = "English",

isbn = "9783642314636",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "PART 1",

pages = "730--739",

booktitle = "Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers",

edition = "PART 1",

note = "9th International Conference on Parallel Processing and Applied Mathematics, PPAM 2011 ; Conference date: 11-09-2011 Through 14-09-2011",

}

Luszczek, P & Dongarra, J 2012, Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling. in Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers. PART 1 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 7203 LNCS, pp. 730-739, 9th International Conference on Parallel Processing and Applied Mathematics, PPAM 2011, Torun, Poland, 09/11/11. https://doi.org/10.1007/978-3-642-31464-3_74

Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling. / Luszczek, Piotr; Dongarra, Jack.
Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers. PART 1. ed. 2012. p. 730-739 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7203 LNCS, No. PART 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling

AU - Luszczek, Piotr

AU - Dongarra, Jack

PY - 2012

Y1 - 2012

N2 - We present a modeling framework to accurately predict time to run dense linear algebra calculation. We report the framework's accuracy in a number of varied computational environments such as shared memory multicore systems, clusters, and large supercomputing installations with tens of thousands of cores. We also test the accuracy for various algorithms, each of which having a different scaling properties and tolerance to low-bandwidth/high-latency interconnects. The predictive accuracy is very good and on the order of measurement accuracy which makes the method suitable for both dedicated and non-dedicated environments. We also present a practical application of our model to reduce the time required to tune and optimize large parallel runs whose time is dominated by linear algebra computations. We show practical examples of how to apply the methodology to avoid common pitfalls and reduce the influence of measurement errors and the inherent performance variability.

AB - We present a modeling framework to accurately predict time to run dense linear algebra calculation. We report the framework's accuracy in a number of varied computational environments such as shared memory multicore systems, clusters, and large supercomputing installations with tens of thousands of cores. We also test the accuracy for various algorithms, each of which having a different scaling properties and tolerance to low-bandwidth/high-latency interconnects. The predictive accuracy is very good and on the order of measurement accuracy which makes the method suitable for both dedicated and non-dedicated environments. We also present a practical application of our model to reduce the time required to tune and optimize large parallel runs whose time is dominated by linear algebra computations. We show practical examples of how to apply the methodology to avoid common pitfalls and reduce the influence of measurement errors and the inherent performance variability.

KW - Linear systems

KW - modeling techniques

KW - parallel algorithms

UR - http://www.scopus.com/inward/record.url?scp=84865217566&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-31464-3_74

DO - 10.1007/978-3-642-31464-3_74

M3 - Conference contribution

AN - SCOPUS:84865217566

SN - 9783642314636

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 730

EP - 739

BT - Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers

T2 - 9th International Conference on Parallel Processing and Applied Mathematics, PPAM 2011

Y2 - 11 September 2011 through 14 September 2011

ER -

Luszczek P, Dongarra J. Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling. In Parallel Processing and Applied Mathematics - 9th International Conference, PPAM 2011, Revised Selected Papers. PART 1 ed. 2012. p. 730-739. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). doi: 10.1007/978-3-642-31464-3_74

Reducing the time to tune parallel dense linear algebra routines with partial execution and performance modeling

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this