Optimization and parallelization of the thermal-hydraulic subchannel code CTF for high-fidelity multi-physics applications

Robert K. Salko, Rodney C. Schmidt, Maria N. Avramova

Research output: Contribution to journalArticlepeer-review

48 Scopus citations

Abstract

This paper describes major improvements to the computational infrastructure of the CTF subchannel code so that full-core, pincell-resolved (i.e., one computational subchannel per real bundle flow channel) simulations can now be performed in much shorter run-times, either in stand-alone mode or as part of coupled-code multi-physics calculations. These improvements support the goals of the Department Of Energy Consortium for Advanced Simulation of Light Water Reactors (CASL) Energy Innovation Hub to develop high fidelity multi-physics simulation tools for nuclear energy design and analysis. A set of serial code optimizations - including fixing computational inefficiencies, optimizing the numerical approach, and making smarter data storage choices - are first described and shown to reduce both execution time and memory usage by about a factor of ten. Next, a "single program multiple data" parallelization strategy targeting distributed memory "multiple instruction multiple data" platforms utilizing domain decomposition is presented. In this approach, data communication between processors is accomplished by inserting standard Message-Passing Interface (MPI) calls at strategic points in the code. The domain decomposition approach implemented assigns one MPI process to each fuel assembly, with each domain being represented by its own CTF input file. The creation of CTF input files, both for serial and parallel runs, is also fully automated through use of a pressurized water reactor (PWR) pre-processor utility that uses a greatly simplified set of user input compared with the traditional CTF input. To run CTF in parallel, two additional libraries are currently needed: MPI, for inter-processor message passing, and the Parallel Extensible Toolkit for Scientific Computation (PETSc), which is used to solve the global pressure matrix in parallel. Results presented include a set of testing and verification calculations and performance tests assessing parallel scaling characteristics up to a full-core, pincell-resolved model of a PWR core containing 193 17 × 17 assemblies under hot full-power conditions. This model, representative of Watts Bar Unit 1 and containing about 56,000 pins, was modeled with roughly 59,000 subchannels, leading to about 2.8 million thermal-hydraulic control volumes in total. Results demonstrate that CTF can now perform full-core analysis of a PWR (not previously possible owing to excessively long runtimes and memory requirements) on the order of 20 min. This new capability not only is useful to stand-alone CTF users, but also is being leveraged in support of coupled code multi-physics calculations being done in the CASL program.

Original languageEnglish
Pages (from-to)122-130
Number of pages9
JournalAnnals of Nuclear Energy
Volume84
DOIs
StatePublished - Jul 28 2015

Funding

This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the US Department of Energy under Contract No. DE-AC05-00OR22725 . This research was supported by the Consortium for Advanced Simulation of Light Water Reactors ( www.casl.gov ), an Energy Innovation Hub ( http://www.energy.gov/hubs ) for Modeling and Simulation of Nuclear Reactors under US Department of Energy Contract No. DE-AC05-00OR22725 .

FundersFunder number
Consortium for Advanced Simulation of Light Water Reactors
Energy Innovation Hub
Modeling and Simulation of Nuclear Reactors
U.S. Department of EnergyDE-AC05-00OR22725
Office of Science

    Keywords

    • COBRA-TF
    • CTF
    • MPI
    • Parallel
    • Subchannel

    Fingerprint

    Dive into the research topics of 'Optimization and parallelization of the thermal-hydraulic subchannel code CTF for high-fidelity multi-physics applications'. Together they form a unique fingerprint.

    Cite this