TY - GEN
T1 - Towards Cross-Facility Workflows Orchestration through Distributed Automation
AU - Skluzacek, Tyler J.
AU - Souza, Renan
AU - Coletti, Mark
AU - Suter, Frederic
AU - Ferreira Da Silva, Rafael
N1 - Publisher Copyright:
© 2024 ACM.
PY - 2024/7/17
Y1 - 2024/7/17
N2 - Modern science relies on end-to-end workflows that incorporate experimental instruments and utilize edge, cloud, or high-performance computing and storage resources. These components are geographically dispersed across various user facilities and interconnected through high-speed networks. In this paper, we present Zambeze, an automated distributed framework designed to facilitate this new class of cross-facility workflows. Utilizing swarm intelligence principles, Zambeze orchestrates science campaigns by managing distributed autonomous agents. These agents can offer a suite of services, including computing, storage, and data management. We demonstrate the feasibility of Zambeze through a real-world application involving electron microscopy, enhanced with Artificial Intelligence capabilities.
AB - Modern science relies on end-to-end workflows that incorporate experimental instruments and utilize edge, cloud, or high-performance computing and storage resources. These components are geographically dispersed across various user facilities and interconnected through high-speed networks. In this paper, we present Zambeze, an automated distributed framework designed to facilitate this new class of cross-facility workflows. Utilizing swarm intelligence principles, Zambeze orchestrates science campaigns by managing distributed autonomous agents. These agents can offer a suite of services, including computing, storage, and data management. We demonstrate the feasibility of Zambeze through a real-world application involving electron microscopy, enhanced with Artificial Intelligence capabilities.
KW - distributed computing
KW - heterogeneous computing
KW - workflows
UR - http://www.scopus.com/inward/record.url?scp=85200354544&partnerID=8YFLogxK
U2 - 10.1145/3626203.3670606
DO - 10.1145/3626203.3670606
M3 - Conference contribution
AN - SCOPUS:85200354544
T3 - PEARC 2024 - Practice and Experience in Advanced Research Computing 2024: Human Powered Computing
BT - PEARC 2024 - Practice and Experience in Advanced Research Computing 2024
PB - Association for Computing Machinery, Inc
T2 - 2024 Practice and Experience in Advanced Research Computing, PEARC 2024
Y2 - 21 July 2024 through 25 July 2024
ER -