Processing full-scale square kilometre array data on the summit supercomputer

Ruonan Wang, Rodrigo Tobar, Markus Dolensky, Tao An, Andreas Wicenec, Chen Wu, Fred Dulwich, Norbert Podhorszki, Valentine Anantharaj, Eric Suchyta, Baoqiang Lao, Scott Klasky

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

8 Scopus citations

Abstract

This work presents a workflow for simulating and processing the full-scale low-frequency telescope data of the Square Kilometre Array (SKA) Phase 1. The SKA project will enter the construction phase soon, and once completed, it will be the world's largest radio telescope and one of the world's largest data generators. The authors used Summit to mimic an endto-end SKA workflow, simulating a dataset of a typical 6 hour observation and then processing that dataset with an imaging pipeline. This workflow was deployed and run on 4,560 compute nodes, and used 27,360 GPUs to generate 2.6 PB of data. This was the first time that radio astronomical data were processed at this scale. Results show that the workflow has the capability to process one of the key SKA science cases, an Epoch of Reionization observation. This analysis also helps reveal critical design factors for the next-generation radio telescopes and the required dedicated processing facilities.

Original languageEnglish
Title of host publicationProceedings of SC 2020
Subtitle of host publicationInternational Conference for High Performance Computing, Networking, Storage and Analysis
PublisherIEEE Computer Society
ISBN (Electronic)9781728199986
DOIs
StatePublished - Nov 2020
Event2020 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020 - Virtual, Atlanta, United States
Duration: Nov 9 2020Nov 19 2020

Publication series

NameInternational Conference for High Performance Computing, Networking, Storage and Analysis, SC
Volume2020-November
ISSN (Print)2167-4329
ISSN (Electronic)2167-4337

Conference

Conference2020 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020
Country/TerritoryUnited States
CityVirtual, Atlanta
Period11/9/2011/19/20

Funding

This work was jointly sponsored by the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, the US Department of Energy’s Scientific D iscovery T hrough A dvanced C omputing ( SciDAC) p roject, the Australian Department of Industry, Innovation and Science through SKA Bridging grant 75656, and China SKA Regional Center prototype funded by Ministry of Science and Technology of the People’s Republic of China and Chinese Academy of Sciences through SKA grant 2018YFA0404603. This work was jointly sponsored by the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, the US Department of Energy’s Scientific Discovery Through Advanced Computing (SciDAC) RAPIDS project, and the Australian Department of Industry, Innovation, and Science through SKA Bridging grant 75656. This research used resources of the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory, which is supported by the US Department of Energy’s Office of Science under Contract No. DE-AC05-00OR22725. This work used resources of China SKA Regional Centre prototype [30] funded by the National Key R&D Programme of China (2018YFA0404603) and Chinese Academy of Sciences (114231KYSB20170003). Preliminary testing used the Tianhe-2 supercomputer at the National Supercomputer Center in Guangzhou, China.

Keywords

  • ADIOS2
  • DALiuGE
  • OSKAR2
  • SKA
  • Summit
  • extreme-scale workflow

Fingerprint

Dive into the research topics of 'Processing full-scale square kilometre array data on the summit supercomputer'. Together they form a unique fingerprint.

Cite this