The ICON-A model for direct QBO simulations on GPUs (version icon-cscs:baf28a514)

Marco A. Giorgetta, William Sawyer, Xavier Lapillonne, Panagiotis Adamidis, Dmitry Alexeev, Valentin Clément, Remo Dietlicher, Jan Frederik Engels, Monika Esch, Henning Franke, Claudia Frauen, Walter M. Hannah, Benjamin R. Hillman, Luis Kornblueh, Philippe Marti, Matthew R. Norman, Robert Pincus, Sebastian Rast, Daniel Reinert, Reiner SchnurUwe Schulzweida, Bjorn Stevens

Research output: Contribution to journalArticlepeer-review

15 Scopus citations

Abstract

Classical numerical models for the global atmosphere, as used for numerical weather forecasting or climate research, have been developed for conventional central processing unit (CPU) architectures. This hinders the employment of such models on current top-performing supercomputers, which achieve their computing power with hybrid architectures, mostly using graphics processing units (GPUs). Thus also scientific applications of such models are restricted to the lesser computer power of CPUs. Here we present the development of a GPU-enabled version of the ICON atmosphere model (ICON-A), motivated by a research project on the quasi-biennial oscillation (QBO), a global-scale wind oscillation in the equatorial stratosphere that depends on a broad spectrum of atmospheric waves, which originates from tropical deep convection. Resolving the relevant scales, from a few kilometers to the size of the globe, is a formidable computational problem, which can only be realized now on top-performing supercomputers. This motivated porting ICON-A, in the specific configuration needed for the research project, in a first step to the GPU architecture of the Piz Daint computer at the Swiss National Supercomputing Centre and in a second step to the JUWELS Booster computer at the Forschungszentrum Jülich. On Piz Daint, the ported code achieves a single-node GPU vs. CPU speedup factor of 6.4 and allows for global experiments at a horizontal resolution of 5 km on 1024 computing nodes with 1 GPU per node with a turnover of 48 simulated days per day. On JUWELS Booster, the more modern hardware in combination with an upgraded code base allows for simulations at the same resolution on 128 computing nodes with 4 GPUs per node and a turnover of 133 simulated days per day. Additionally, the code still remains functional on CPUs, as is demonstrated by additional experiments on the Levante compute system at the German Climate Computing Center. While the application shows good weak scaling over the tested 16-fold increase in grid size and node count, making also higher resolved global simulations possible, the strong scaling on GPUs is relatively poor, which limits the options to increase turnover with more nodes. Initial experiments demonstrate that the ICON-A model can simulate downward-propagating QBO jets, which are driven by wave-mean flow interaction.

Original languageEnglish
Pages (from-to)6985-7016
Number of pages32
JournalGeoscientific Model Development
Volume15
Issue number18
DOIs
StatePublished - Sep 16 2022

Funding

Philippe Marti and Valentin Clément received funding from the Platform for Advanced Scientific Computing (PASC) of ETH (reference number 2017-8). Some portions of the work of Matthew R. Norman, Benjamin R. Hillman, and Walter M. Hannah on RTE+RRTMGP were funded by the US Department of Energy (grant no. DE-SC0021262) and by Lawrence Livermore National Laboratory (contract no. DE-AC52-07NA27344).The article processing charges for this open-access publication were covered by the Max Planck Society.

FundersFunder number
U.S. Department of EnergyDE-SC0021262
Lawrence Livermore National LaboratoryDE-AC52-07NA27344

    Fingerprint

    Dive into the research topics of 'The ICON-A model for direct QBO simulations on GPUs (version icon-cscs:baf28a514)'. Together they form a unique fingerprint.

    Cite this