TY - GEN
T1 - Near real-Time analysis of big fusion data on HPC systems
AU - Kube, Ralph
AU - Churchill, R. Michael
AU - Choi, Jong
AU - Wang, Ruonan
AU - Choi, Minjun
AU - Klasky, Scott
AU - Chang, C. S.
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/11
Y1 - 2020/11
N2 - We are developing the Delta framework that aims to tackle big-data challenges specific to fusion energy sciences. Delta can be used to connect fusion experiments to remote supercomputers. Streaming measurements to distributed compute resources allows to automatically perform high-dimensional data analysis on a cadence that exceeds experimental schedules. Making data analysis results available before the next experiments allows scientists to make more informed decisions about configuration of upcoming experiments. Here we describe how Delta uses database and virtualization facilities, as well as high-performance computing, at the National Energy Research Compute Center to offer a vertically integrated near real-Time data analysis and visualization. We also report on ongoing efforts to port the data analysis part of Delta to graphical processing units, which show a reduction of the analysis wall-Time for a benchmark workflow by about 35% when compared to a serial implementation.
AB - We are developing the Delta framework that aims to tackle big-data challenges specific to fusion energy sciences. Delta can be used to connect fusion experiments to remote supercomputers. Streaming measurements to distributed compute resources allows to automatically perform high-dimensional data analysis on a cadence that exceeds experimental schedules. Making data analysis results available before the next experiments allows scientists to make more informed decisions about configuration of upcoming experiments. Here we describe how Delta uses database and virtualization facilities, as well as high-performance computing, at the National Energy Research Compute Center to offer a vertically integrated near real-Time data analysis and visualization. We also report on ongoing efforts to port the data analysis part of Delta to graphical processing units, which show a reduction of the analysis wall-Time for a benchmark workflow by about 35% when compared to a serial implementation.
UR - http://www.scopus.com/inward/record.url?scp=85099552960&partnerID=8YFLogxK
U2 - 10.1109/UrgentHPC51945.2020.00012
DO - 10.1109/UrgentHPC51945.2020.00012
M3 - Conference contribution
AN - SCOPUS:85099552960
T3 - Proceedings of UrgentHPC 2020: 2020 International Workshops on Urgent and Interactive HPC, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis
SP - 55
EP - 63
BT - Proceedings of UrgentHPC 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE/ACM International Workshops on Urgent and Interactive HPC, UrgentHPC 2020
Y2 - 13 November 2020
ER -