Accelerated numerical modeling of shallow water flows with MPI, OpenACC, and GPUs

In this paper, a time-explicit Finite-Volume method is adopted to solve the 2-D shallow water equations on an unstructured triangular mesh, using a two-stage Runge-Kutta integrator and a monotone MUSCL model to achieve second-order accuracy in time and space, respectively. A multi-GPU model is presented that uses the Message Passing Interface (MPI) with OpenACC and uses the METIS library to produce the domain decomposition. A CUDA-aware MPI library (GPUDirect) and overlapped MPI communication with computation are used to improve parallel performance. Two benchmark tests with wet and dry downstream beds are used to test the code's accuracy. Good results were achieved compared to the numerical simulations of published studies. Compared with the multi-CPU version of a 6-core CPU, maximum speedups of 56.18 and 331.51 were obtained using a single GPU and 8 GPUs, respectively. Higher mesh resolution enhances acceleration performance, and the model is applicable to other environmental modeling activities.