Towards Cross-Facility Workflows Orchestration through Distributed Automation

Tyler J. Skluzacek, Renan Souza, Mark Coletti, Frederic Suter, Rafael Ferreira Da Silva

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Modern science relies on end-to-end workflows that incorporate experimental instruments and utilize edge, cloud, or high-performance computing and storage resources. These components are geographically dispersed across various user facilities and interconnected through high-speed networks. In this paper, we present Zambeze, an automated distributed framework designed to facilitate this new class of cross-facility workflows. Utilizing swarm intelligence principles, Zambeze orchestrates science campaigns by managing distributed autonomous agents. These agents can offer a suite of services, including computing, storage, and data management. We demonstrate the feasibility of Zambeze through a real-world application involving electron microscopy, enhanced with Artificial Intelligence capabilities.

Original languageEnglish
Title of host publicationPEARC 2024 - Practice and Experience in Advanced Research Computing 2024
Subtitle of host publicationHuman Powered Computing
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9798400704192
DOIs
StatePublished - Jul 17 2024
Event2024 Practice and Experience in Advanced Research Computing, PEARC 2024 - Providence, United States
Duration: Jul 21 2024Jul 25 2024

Publication series

NamePEARC 2024 - Practice and Experience in Advanced Research Computing 2024: Human Powered Computing

Conference

Conference2024 Practice and Experience in Advanced Research Computing, PEARC 2024
Country/TerritoryUnited States
CityProvidence
Period07/21/2407/25/24

Keywords

  • distributed computing
  • heterogeneous computing
  • workflows

Fingerprint

Dive into the research topics of 'Towards Cross-Facility Workflows Orchestration through Distributed Automation'. Together they form a unique fingerprint.

Cite this