Optimizing a multiple right-hand side Dslash kernel for Intel knights corner

Aaron Walden; Sabbir Khan; Bálint Joó; Desh Ranjan; Mohammad Zubair

doi:10.1007/978-3-319-46079-6_28

Optimizing a multiple right-hand side Dslash kernel for Intel knights corner

Aaron Walden
, Sabbir Khan
, Bálint Joó
, Desh Ranjan
, Mohammad Zubair

Advanced Computing for Nuclear, Particle

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

There is a significant interest in the computational physics community to perform lattice quantum chromodynamics (LQCD) simulations, which can run into the trillions of operations. LQCD computations solve a sparse linear system using a Wilson Dslash kernel, which has an arithmetic intensity of 0.88–2.29. This makes Dslash memory bandwidth-bound on most architectures, including Intel Xeon Phi Knights Corner (KNC). Most research optimizing the Dslash operator has been focused on single right-hand side (SRHS) linear solvers. There is a class of LQCD computations which aims to solve systems with multiple right-hand sides (MRHS), presenting additional opportunities for data reuse and vectorization. We present two approaches to MRHS Dslash: a vector register blocking approach and one using the software package QPhiX with a custom code generator for low-level intrinsics.We observed significant speedups using our approaches, with sustained performance of over 700 GFLOPS (single precision) in one instance. We achieved up to 29% of theoretical peak performance compared to a maximum of 13% obtained by the previous SRHS method using QPhiX.

Original language	English
Title of host publication	High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected
Editors	Bernd Mohr, Julian M. Kunkel, Michela Taufer
Publisher	Springer Verlag
Pages	390-401
Number of pages	12
ISBN (Print)	9783319460789
DOIs	https://doi.org/10.1007/978-3-319-46079-6_28
State	Published - 2016
Event	International Workshops on High Performance Computing, ISC High Performance 2016 and Workshop on 2nd International Workshop on Communication Architectures at Extreme Scale, ExaComm 2016, Workshop on Exascale Multi/Many Core Computing Systems, E-MuCoCoS 2016, HPC I/O in the Data Center, HPC-IODC 2016, Application Performance on Intel Xeon Phi – Being Prepared for KNL and Beyond, IXPUG 2016, International Workshop on OpenPOWER for HPC, IWOPH 2016, International Workshop on Performance Portable Programming Models for Accelerators, P^3MA 2016, Workshop on Virtualization in High-Performance Cloud Computing, VHPC 2016, Workshop on Performance and Scalability of Storage Systems, WOPSSS 2016 - Frankfurt, Germany Duration: Jun 19 2016 → Jun 23 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9945 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	International Workshops on High Performance Computing, ISC High Performance 2016 and Workshop on 2nd International Workshop on Communication Architectures at Extreme Scale, ExaComm 2016, Workshop on Exascale Multi/Many Core Computing Systems, E-MuCoCoS 2016, HPC I/O in the Data Center, HPC-IODC 2016, Application Performance on Intel Xeon Phi – Being Prepared for KNL and Beyond, IXPUG 2016, International Workshop on OpenPOWER for HPC, IWOPH 2016, International Workshop on Performance Portable Programming Models for Accelerators, P^3MA 2016, Workshop on Virtualization in High-Performance Cloud Computing, VHPC 2016, Workshop on Performance and Scalability of Storage Systems, WOPSSS 2016
Country/Territory	Germany
City	Frankfurt
Period	06/19/16 → 06/23/16

Funding

Notice: Authored by Jefferson Science Associates, LLC under U.S. DOE Contract No. DE-AC05-06OR23177. The U.S. Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce this manuscript for U.S. Government purposes. This work was partially supported by a grant from Jefferson Lab. Aaron Walden and Sabbir Khan were also partially supported by the Old Dominion University Modeling and Simulation Fellowship Program and gratefully acknowledge this support. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Nuclear Physics under contract DE-AC05-06OR23177.

Keywords

Code generator
LQCD
Optimization
Parallel programming
Performance
Vectorization
Wilson-Dslash
Xeon Phi Knights Corner

Access to Document

10.1007/978-3-319-46079-6_28

Cite this

Walden, A., Khan, S., Joó, B., Ranjan, D., & Zubair, M. (2016). Optimizing a multiple right-hand side Dslash kernel for Intel knights corner. In B. Mohr, J. M. Kunkel, & M. Taufer (Eds.), High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected (pp. 390-401). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9945 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46079-6_28

Walden, Aaron ; Khan, Sabbir ; Joó, Bálint et al. / Optimizing a multiple right-hand side Dslash kernel for Intel knights corner. High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected. editor / Bernd Mohr ; Julian M. Kunkel ; Michela Taufer. Springer Verlag, 2016. pp. 390-401 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d8c240eaf43b430da0981bd4d46ea232,

title = "Optimizing a multiple right-hand side Dslash kernel for Intel knights corner",

abstract = "There is a significant interest in the computational physics community to perform lattice quantum chromodynamics (LQCD) simulations, which can run into the trillions of operations. LQCD computations solve a sparse linear system using a Wilson Dslash kernel, which has an arithmetic intensity of 0.88–2.29. This makes Dslash memory bandwidth-bound on most architectures, including Intel Xeon Phi Knights Corner (KNC). Most research optimizing the Dslash operator has been focused on single right-hand side (SRHS) linear solvers. There is a class of LQCD computations which aims to solve systems with multiple right-hand sides (MRHS), presenting additional opportunities for data reuse and vectorization. We present two approaches to MRHS Dslash: a vector register blocking approach and one using the software package QPhiX with a custom code generator for low-level intrinsics.We observed significant speedups using our approaches, with sustained performance of over 700 GFLOPS (single precision) in one instance. We achieved up to 29\% of theoretical peak performance compared to a maximum of 13\% obtained by the previous SRHS method using QPhiX.",

keywords = "Code generator, LQCD, Optimization, Parallel programming, Performance, Vectorization, Wilson-Dslash, Xeon Phi Knights Corner",

author = "Aaron Walden and Sabbir Khan and B{\'a}lint Jo{\'o} and Desh Ranjan and Mohammad Zubair",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; International Workshops on High Performance Computing, ISC High Performance 2016 and Workshop on 2nd International Workshop on Communication Architectures at Extreme Scale, ExaComm 2016, Workshop on Exascale Multi/Many Core Computing Systems, E-MuCoCoS 2016, HPC I/O in the Data Center, HPC-IODC 2016, Application Performance on Intel Xeon Phi – Being Prepared for KNL and Beyond, IXPUG 2016, International Workshop on OpenPOWER for HPC, IWOPH 2016, International Workshop on Performance Portable Programming Models for Accelerators, P\textasciicircum{}3MA 2016, Workshop on Virtualization in High-Performance Cloud Computing, VHPC 2016, Workshop on Performance and Scalability of Storage Systems, WOPSSS 2016 ; Conference date: 19-06-2016 Through 23-06-2016",

year = "2016",

doi = "10.1007/978-3-319-46079-6\_28",

language = "English",

isbn = "9783319460789",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "390--401",

editor = "Bernd Mohr and Kunkel, \{Julian M.\} and Michela Taufer",

booktitle = "High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P\textasciicircum{}3MA, VHPC, WOPSSS, Revised Selected",

}

Walden, A, Khan, S, Joó, B, Ranjan, D & Zubair, M 2016, Optimizing a multiple right-hand side Dslash kernel for Intel knights corner. in B Mohr, JM Kunkel & M Taufer (eds), High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9945 LNCS, Springer Verlag, pp. 390-401, International Workshops on High Performance Computing, ISC High Performance 2016 and Workshop on 2nd International Workshop on Communication Architectures at Extreme Scale, ExaComm 2016, Workshop on Exascale Multi/Many Core Computing Systems, E-MuCoCoS 2016, HPC I/O in the Data Center, HPC-IODC 2016, Application Performance on Intel Xeon Phi – Being Prepared for KNL and Beyond, IXPUG 2016, International Workshop on OpenPOWER for HPC, IWOPH 2016, International Workshop on Performance Portable Programming Models for Accelerators, P^3MA 2016, Workshop on Virtualization in High-Performance Cloud Computing, VHPC 2016, Workshop on Performance and Scalability of Storage Systems, WOPSSS 2016, Frankfurt, Germany, 06/19/16. https://doi.org/10.1007/978-3-319-46079-6_28

Optimizing a multiple right-hand side Dslash kernel for Intel knights corner. / Walden, Aaron; Khan, Sabbir; Joó, Bálint et al.
High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected. ed. / Bernd Mohr; Julian M. Kunkel; Michela Taufer. Springer Verlag, 2016. p. 390-401 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9945 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Optimizing a multiple right-hand side Dslash kernel for Intel knights corner

AU - Walden, Aaron

AU - Khan, Sabbir

AU - Joó, Bálint

AU - Ranjan, Desh

AU - Zubair, Mohammad

N1 - Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016

Y1 - 2016

N2 - There is a significant interest in the computational physics community to perform lattice quantum chromodynamics (LQCD) simulations, which can run into the trillions of operations. LQCD computations solve a sparse linear system using a Wilson Dslash kernel, which has an arithmetic intensity of 0.88–2.29. This makes Dslash memory bandwidth-bound on most architectures, including Intel Xeon Phi Knights Corner (KNC). Most research optimizing the Dslash operator has been focused on single right-hand side (SRHS) linear solvers. There is a class of LQCD computations which aims to solve systems with multiple right-hand sides (MRHS), presenting additional opportunities for data reuse and vectorization. We present two approaches to MRHS Dslash: a vector register blocking approach and one using the software package QPhiX with a custom code generator for low-level intrinsics.We observed significant speedups using our approaches, with sustained performance of over 700 GFLOPS (single precision) in one instance. We achieved up to 29% of theoretical peak performance compared to a maximum of 13% obtained by the previous SRHS method using QPhiX.

AB - There is a significant interest in the computational physics community to perform lattice quantum chromodynamics (LQCD) simulations, which can run into the trillions of operations. LQCD computations solve a sparse linear system using a Wilson Dslash kernel, which has an arithmetic intensity of 0.88–2.29. This makes Dslash memory bandwidth-bound on most architectures, including Intel Xeon Phi Knights Corner (KNC). Most research optimizing the Dslash operator has been focused on single right-hand side (SRHS) linear solvers. There is a class of LQCD computations which aims to solve systems with multiple right-hand sides (MRHS), presenting additional opportunities for data reuse and vectorization. We present two approaches to MRHS Dslash: a vector register blocking approach and one using the software package QPhiX with a custom code generator for low-level intrinsics.We observed significant speedups using our approaches, with sustained performance of over 700 GFLOPS (single precision) in one instance. We achieved up to 29% of theoretical peak performance compared to a maximum of 13% obtained by the previous SRHS method using QPhiX.

KW - Code generator

KW - LQCD

KW - Optimization

KW - Parallel programming

KW - Performance

KW - Vectorization

KW - Wilson-Dslash

KW - Xeon Phi Knights Corner

UR - https://www.scopus.com/pages/publications/84992672585

U2 - 10.1007/978-3-319-46079-6_28

DO - 10.1007/978-3-319-46079-6_28

M3 - Conference contribution

AN - SCOPUS:84992672585

SN - 9783319460789

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 390

EP - 401

BT - High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected

A2 - Mohr, Bernd

A2 - Kunkel, Julian M.

A2 - Taufer, Michela

PB - Springer Verlag

T2 - International Workshops on High Performance Computing, ISC High Performance 2016 and Workshop on 2nd International Workshop on Communication Architectures at Extreme Scale, ExaComm 2016, Workshop on Exascale Multi/Many Core Computing Systems, E-MuCoCoS 2016, HPC I/O in the Data Center, HPC-IODC 2016, Application Performance on Intel Xeon Phi – Being Prepared for KNL and Beyond, IXPUG 2016, International Workshop on OpenPOWER for HPC, IWOPH 2016, International Workshop on Performance Portable Programming Models for Accelerators, P^3MA 2016, Workshop on Virtualization in High-Performance Cloud Computing, VHPC 2016, Workshop on Performance and Scalability of Storage Systems, WOPSSS 2016

Y2 - 19 June 2016 through 23 June 2016

ER -

Walden A, Khan S, Joó B, Ranjan D, Zubair M. Optimizing a multiple right-hand side Dslash kernel for Intel knights corner. In Mohr B, Kunkel JM, Taufer M, editors, High Performance Computing - ISC High Performance 2016 International Workshops ExaComm, E-MuCoCoS, HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, WOPSSS, Revised Selected. Springer Verlag. 2016. p. 390-401. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-46079-6_28

Optimizing a multiple right-hand side Dslash kernel for Intel knights corner

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this