TY - GEN
T1 - Calvera
T2 - Smoky Mountains Computational Sciences and Engineering Conference, SMC 2022
AU - Watson, Gregory R.
AU - Cage, Gregory
AU - Fortney, Jon
AU - Granroth, Garrett E.
AU - Hughes, Harry
AU - Maier, Thomas
AU - McDonnell, Marshall
AU - Ramirez-Cuesta, Anibal
AU - Smith, Robert
AU - Yakubov, Sergey
AU - Zhou, Wenduo
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - Data analysis for neutron scattering experiments is driven by the scientific needs of the instrument users and varies greatly by technique and field of study. Data from an experiment must first be “reduced” so that instrument artifacts are removed, and then scientists must choose from a wide variety of tools and applications to assemble a workflow that enables useful scientific results to be extracted. The highly manual nature of this process, combined with difficulty accessing computational resources and data when needed, puts limits on the efficiency and nature of the analysis undertaken. In addition, other activities, such as tracking data provenance to ensure the analysis is reproducible, or providing live data analysis during experiment runs, are also difficult to achieve. Calvera is a platform that aims to solve many of the difficulties encountered by scientists as they analyze experimental neutron scattering data. In particular, the platform will provide an integration point for a range of services, such as data virtualization, remote computation, and visualization under the control of a workflow management system. In addition, the platform will handle security related issues, and maintain a history of the data sets employed during workflow execution. User’s will be able to construct, manage, and share workflows via a graphical user interface, as well as script workflows via a python API. In this paper, we will describe the architecture and design of Calvera, as well as how we will address the many requirements for executing neutron science workflows in a distributed environment.
AB - Data analysis for neutron scattering experiments is driven by the scientific needs of the instrument users and varies greatly by technique and field of study. Data from an experiment must first be “reduced” so that instrument artifacts are removed, and then scientists must choose from a wide variety of tools and applications to assemble a workflow that enables useful scientific results to be extracted. The highly manual nature of this process, combined with difficulty accessing computational resources and data when needed, puts limits on the efficiency and nature of the analysis undertaken. In addition, other activities, such as tracking data provenance to ensure the analysis is reproducible, or providing live data analysis during experiment runs, are also difficult to achieve. Calvera is a platform that aims to solve many of the difficulties encountered by scientists as they analyze experimental neutron scattering data. In particular, the platform will provide an integration point for a range of services, such as data virtualization, remote computation, and visualization under the control of a workflow management system. In addition, the platform will handle security related issues, and maintain a history of the data sets employed during workflow execution. User’s will be able to construct, manage, and share workflows via a graphical user interface, as well as script workflows via a python API. In this paper, we will describe the architecture and design of Calvera, as well as how we will address the many requirements for executing neutron science workflows in a distributed environment.
KW - Data analysis
KW - Data management
KW - Distributed computing
KW - Ecosystem
KW - Neutron science
KW - Visualization
KW - Workflows
UR - http://www.scopus.com/inward/record.url?scp=85148689346&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-23606-8_9
DO - 10.1007/978-3-031-23606-8_9
M3 - Conference contribution
AN - SCOPUS:85148689346
SN - 9783031236051
T3 - Communications in Computer and Information Science
SP - 137
EP - 154
BT - Accelerating Science and Engineering Discoveries Through Integrated Research Infrastructure for Experiment, Big Data, Modeling and Simulation - 22nd Smoky Mountains Computational Sciences and Engineering Conference, SMC 2022, Revised Selected Papers
A2 - Doug, Kothe
A2 - Al, Geist
A2 - Pophale, Swaroop
A2 - Liu, Hong
A2 - Parete-Koon, Suzanne
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 24 August 2022 through 25 August 2022
ER -