The Evolution of the Pegasus Workflow Management Software

Ewa Deelman; Karan Vahi; Mats Rynge; Rajiv Mayani; Rafael Ferreira Da Silva; George Papadimitriou; Miron Livny

doi:10.1109/MCSE.2019.2919690

The Evolution of the Pegasus Workflow Management Software

Ewa Deelman, Karan Vahi, Mats Rynge, Rajiv Mayani, Rafael Ferreira Da Silva, George Papadimitriou, Miron Livny

Workflow and Ecosystem Services

Research output: Contribution to journal › Article › peer-review

78 Scopus citations

Abstract

Since 2001, the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-Throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development, it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. It concludes with selected lessons learned.

Original language	English
Article number	8725518
Pages (from-to)	22-36
Number of pages	15
Journal	Computing in Science and Engineering
Volume	21
Issue number	4
DOIs	https://doi.org/10.1109/MCSE.2019.2919690
State	Published - Jul 1 2019

Funding

This work was supported by NSF Contract #1664162, “SI2-SSI: Pegasus: Automating Compute and Data Intensive Science,” and Contract #1148515, “SI2-SSI: Distributed Workflow Management Research and Software in Support of Science”; and partly funded by DOE contract

Access to Document

10.1109/MCSE.2019.2919690

Cite this

@article{1444df4ca9214c62bdb2a53c37e486d3,

title = "The Evolution of the Pegasus Workflow Management Software",

abstract = "Since 2001, the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-Throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development, it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. It concludes with selected lessons learned.",

author = "Ewa Deelman and Karan Vahi and Mats Rynge and Rajiv Mayani and {Da Silva}, {Rafael Ferreira} and George Papadimitriou and Miron Livny",

note = "Publisher Copyright: {\textcopyright} 1999-2011 IEEE.",

year = "2019",

month = jul,

day = "1",

doi = "10.1109/MCSE.2019.2919690",

language = "English",

volume = "21",

pages = "22--36",

journal = "Computing in Science and Engineering",

issn = "1521-9615",

publisher = "Institute of Electrical and Electronics Engineers",

number = "4",

}

TY - JOUR

T1 - The Evolution of the Pegasus Workflow Management Software

AU - Deelman, Ewa

AU - Vahi, Karan

AU - Rynge, Mats

AU - Mayani, Rajiv

AU - Da Silva, Rafael Ferreira

AU - Papadimitriou, George

AU - Livny, Miron

PY - 2019/7/1

Y1 - 2019/7/1

N2 - Since 2001, the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-Throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development, it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. It concludes with selected lessons learned.

AB - Since 2001, the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-Throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development, it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. It concludes with selected lessons learned.

UR - http://www.scopus.com/inward/record.url?scp=85067059724&partnerID=8YFLogxK

U2 - 10.1109/MCSE.2019.2919690

DO - 10.1109/MCSE.2019.2919690

M3 - Article

AN - SCOPUS:85067059724

SN - 1521-9615

VL - 21

SP - 22

EP - 36

JO - Computing in Science and Engineering

JF - Computing in Science and Engineering

IS - 4

M1 - 8725518

ER -

The Evolution of the Pegasus Workflow Management Software

Abstract

Funding

Access to Document

Other files and links

Fingerprint

Cite this