Novel Approaches Toward Scalable Composable Workflows in Hyper-Heterogeneous Computing Environments

Jonathan Bader, Jim Belak, Matthew Bement, Matthew Berry, Robert Carson, Daniela Cassol, Stephen Chan, John Coleman, Kastan Day, Alejandro Duque, Kjiersten Fagnan, Jeff Froula, Shantenu Jha, Daniel S. Katz, Piotr Kica, Volodymyr Kindratenko, Edward Kirton, Ramani Kothadia, Daniel Laney, Fabian LehmannUlf Leser, Sabina Licholai, MacIej Malawski, Mario Melara, Elais Player, Matt Rolchigo, Setareh Sarrafan, Seung Jin Sul, Abdullah Syed, Lauritz Thamsen, Mikhail Titov, Matteo Turilli, Silvina Caino-Lores, Anirban Mandal

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

The annual Workshop on Workflows in Support of Large-Scale Science (WORKS) is a premier venue for the scientific workflow community to present the latest advances in research and development on the many facets of scientific workflows throughout their life-cycle. The Lightning Talks at WORKS focus on describing a novel tool, scientific workflow, or concept, which are work-in-progress and address emerging technologies and frameworks to foster discussion in the community. This paper summarizes the lightning talks at the 2023 edition of WORKS, covering five topics: leveraging large language models to build and execute workflows; developing a common workflow scheduler interface; scaling uncertainty workflow applications on exascale computing systems; evaluating a transcriptomics workflow for cloud vs. HPC systems; and best practices in migrating legacy workflows to workflow management systems.

Original languageEnglish
Title of host publicationProceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023
PublisherAssociation for Computing Machinery
Pages2097-2108
Number of pages12
ISBN (Electronic)9798400707858
DOIs
StatePublished - Nov 12 2023
Event2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023 - Denver, United States
Duration: Nov 12 2023Nov 17 2023

Publication series

NameACM International Conference Proceeding Series

Conference

Conference2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023
Country/TerritoryUnited States
CityDenver
Period11/12/2311/17/23

Funding

The materials this work is based upon are supported by NSF award 2050195; the German Research Foundation (DFG) CRC 1404: "FONDA: Foundations of Workflows for Large-Scale Scientific Data Analysis;" the Office of Science of the U.S. Department of Energy (DOE) under Contract No. DE-AC05-00OR22725; the Exascale Computing Project (ECP, 17-SC-20-SC) ExaWorks project under DOE Contract No. DE-SC0012704; Lawrence Livermore National Laboratory under Contract No. DE-AC52-07NA27344; DOE INCITE awards for allocations on Summit, Crusher, and Frontier; the European Union (EU) Horizon 2020 research and innovation programme under grant agreement Sano No. 857533 within the International Research Agendas programme of the Foundation for Polish Science, co-financed by the EU under the European Regional Development Fund; the EU Horizon 2020 research and innovation programme under grant agreement NEARDATA No. 101092644; the PLGrid Infrastructure and the Ares supercomputer at ACC Cyfronet AGH; and DOE Joint Genome Institute under Contract No. DE-AC02-05CH11231.

Keywords

  • Scientific workflows
  • cloud computing
  • high-performance computing
  • large language models
  • legacy workflows
  • workflow scheduling

Fingerprint

Dive into the research topics of 'Novel Approaches Toward Scalable Composable Workflows in Hyper-Heterogeneous Computing Environments'. Together they form a unique fingerprint.

Cite this