One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming

Kshitij Mehta; Maxime Hugues; Oscar Hernandez; David E. Bernholdt; Henri Calandra

doi:10.1109/IPDPS.2017.82

One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming

Kshitij Mehta, Maxime Hugues, Oscar Hernandez, David E. Bernholdt, Henri Calandra

Programming Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

One-Way Wave Equation Migration (OWEM) is a depth migration algorithm used for seismic imaging. A parallel version of this algorithm is widely implemented using MPI. Heterogenous architectures that use GPUs have become popular in the Top 500 because of their performance/power ratio. In this paper, we discuss the methodology and code transformations used to port OWEM to GPUs using OpenACC, along with the code changes needed for scaling the application up to 18,400 GPUs (more than 98%) of the Titan leadership class supercomputer at Oak Ridget National Laboratory. For the individual OpenACC kernels, we achieved an average of 3X speedup on a test dataset using one GPU as compared with an 8-core Intel Sandy Bridge CPU. The application was then run at large scale on the Titan supercomputer achieving a peak of 1.2 petaflops using an average of 5.5 megawatts. After porting the application to GPUs, we discuss how we dealt with other challenges of running at scale such as the application becoming more I/O bound and prone to silent errors. We believe this work will serve as valuable proof that directive-based programming models are a viable option for scaling HPC applications to heterogenous architectures.

Original language	English
Title of host publication	Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	224-233
Number of pages	10
ISBN (Electronic)	9781538639146
DOIs	https://doi.org/10.1109/IPDPS.2017.82
State	Published - Jun 30 2017
Event	31st IEEE International Parallel and Distributed Processing Symposium, IPDPS 2017 - Orlando, United States Duration: May 29 2017 → Jun 2 2017

Publication series

Name	Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017

Conference

Conference	31st IEEE International Parallel and Distributed Processing Symposium, IPDPS 2017
Country/Territory	United States
City	Orlando
Period	05/29/17 → 06/2/17

Funding

We would like to thank Jack Wells, Suzy Tichenor, Don Maxwell, Sarp Oral, and Jason Hill from Oak Ridge National Lab and the National Center for Computational Sciences (NCCS) for their help and support with the project. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Keywords

imaging
one-way
openacc
seismic

Access to Document

10.1109/IPDPS.2017.82

Cite this

Mehta, K., Hugues, M., Hernandez, O., Bernholdt, D. E., & Calandra, H. (2017). One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming. In Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017 (pp. 224-233). Article 7967112 (Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS.2017.82

Mehta, Kshitij ; Hugues, Maxime ; Hernandez, Oscar et al. / One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming. Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 224-233 (Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017).

@inproceedings{073d7731db144272944fb5ae00b72d7a,

title = "One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming",

abstract = "One-Way Wave Equation Migration (OWEM) is a depth migration algorithm used for seismic imaging. A parallel version of this algorithm is widely implemented using MPI. Heterogenous architectures that use GPUs have become popular in the Top 500 because of their performance/power ratio. In this paper, we discuss the methodology and code transformations used to port OWEM to GPUs using OpenACC, along with the code changes needed for scaling the application up to 18,400 GPUs (more than 98%) of the Titan leadership class supercomputer at Oak Ridget National Laboratory. For the individual OpenACC kernels, we achieved an average of 3X speedup on a test dataset using one GPU as compared with an 8-core Intel Sandy Bridge CPU. The application was then run at large scale on the Titan supercomputer achieving a peak of 1.2 petaflops using an average of 5.5 megawatts. After porting the application to GPUs, we discuss how we dealt with other challenges of running at scale such as the application becoming more I/O bound and prone to silent errors. We believe this work will serve as valuable proof that directive-based programming models are a viable option for scaling HPC applications to heterogenous architectures.",

keywords = "imaging, one-way, openacc, seismic",

author = "Kshitij Mehta and Maxime Hugues and Oscar Hernandez and Bernholdt, {David E.} and Henri Calandra",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 31st IEEE International Parallel and Distributed Processing Symposium, IPDPS 2017 ; Conference date: 29-05-2017 Through 02-06-2017",

year = "2017",

month = jun,

day = "30",

doi = "10.1109/IPDPS.2017.82",

language = "English",

series = "Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "224--233",

booktitle = "Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017",

}

Mehta, K, Hugues, M, Hernandez, O , Bernholdt, DE & Calandra, H 2017, One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming. in Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017., 7967112, Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017, Institute of Electrical and Electronics Engineers Inc., pp. 224-233, 31st IEEE International Parallel and Distributed Processing Symposium, IPDPS 2017, Orlando, United States, 05/29/17. https://doi.org/10.1109/IPDPS.2017.82

One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming. / Mehta, Kshitij; Hugues, Maxime; Hernandez, Oscar et al.
Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 224-233 7967112 (Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming

AU - Mehta, Kshitij

AU - Hugues, Maxime

AU - Hernandez, Oscar

AU - Bernholdt, David E.

AU - Calandra, Henri

PY - 2017/6/30

Y1 - 2017/6/30

N2 - One-Way Wave Equation Migration (OWEM) is a depth migration algorithm used for seismic imaging. A parallel version of this algorithm is widely implemented using MPI. Heterogenous architectures that use GPUs have become popular in the Top 500 because of their performance/power ratio. In this paper, we discuss the methodology and code transformations used to port OWEM to GPUs using OpenACC, along with the code changes needed for scaling the application up to 18,400 GPUs (more than 98%) of the Titan leadership class supercomputer at Oak Ridget National Laboratory. For the individual OpenACC kernels, we achieved an average of 3X speedup on a test dataset using one GPU as compared with an 8-core Intel Sandy Bridge CPU. The application was then run at large scale on the Titan supercomputer achieving a peak of 1.2 petaflops using an average of 5.5 megawatts. After porting the application to GPUs, we discuss how we dealt with other challenges of running at scale such as the application becoming more I/O bound and prone to silent errors. We believe this work will serve as valuable proof that directive-based programming models are a viable option for scaling HPC applications to heterogenous architectures.

AB - One-Way Wave Equation Migration (OWEM) is a depth migration algorithm used for seismic imaging. A parallel version of this algorithm is widely implemented using MPI. Heterogenous architectures that use GPUs have become popular in the Top 500 because of their performance/power ratio. In this paper, we discuss the methodology and code transformations used to port OWEM to GPUs using OpenACC, along with the code changes needed for scaling the application up to 18,400 GPUs (more than 98%) of the Titan leadership class supercomputer at Oak Ridget National Laboratory. For the individual OpenACC kernels, we achieved an average of 3X speedup on a test dataset using one GPU as compared with an 8-core Intel Sandy Bridge CPU. The application was then run at large scale on the Titan supercomputer achieving a peak of 1.2 petaflops using an average of 5.5 megawatts. After porting the application to GPUs, we discuss how we dealt with other challenges of running at scale such as the application becoming more I/O bound and prone to silent errors. We believe this work will serve as valuable proof that directive-based programming models are a viable option for scaling HPC applications to heterogenous architectures.

KW - imaging

KW - one-way

KW - openacc

KW - seismic

UR - http://www.scopus.com/inward/record.url?scp=85027725286&partnerID=8YFLogxK

U2 - 10.1109/IPDPS.2017.82

DO - 10.1109/IPDPS.2017.82

M3 - Conference contribution

AN - SCOPUS:85027725286

T3 - Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017

SP - 224

EP - 233

BT - Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 31st IEEE International Parallel and Distributed Processing Symposium, IPDPS 2017

Y2 - 29 May 2017 through 2 June 2017

ER -

Mehta K, Hugues M, Hernandez O , Bernholdt DE, Calandra H. One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming. In Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 224-233. 7967112. (Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium, IPDPS 2017). doi: 10.1109/IPDPS.2017.82

One-Way Wave Equation Migration at Scale on GPUs Using Directive Based Programming

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this