Pegasus in the cloud: Science automation through workflow technologies

Ewa Deelman; Karan Vahi; Mats Rynge; Gideon Juve; Rajiv Mayani; Rafael Ferreira Da Silva

doi:10.1109/MIC.2016.15

Pegasus in the cloud: Science automation through workflow technologies

Ewa Deelman
, Karan Vahi
, Mats Rynge
, Gideon Juve
, Rajiv Mayani
, Rafael Ferreira Da Silva

Research output: Contribution to journal › Article › peer-review

55 Scopus citations

Abstract

The Pegasus Workflow Management System maps abstract, resource-independent workflow descriptions onto distributed computing resources. Pegasus workflows are portable across different infrastructures, optimizable for performance and efficiency, and automatically map to many different storage systems and data flows. Individual researchers can use astronomy workflows, such as science-grade image mosaicking, to find new celestial objects, or collaborate to generate large data resources that are useful to the entire astronomy community. Individual researchers might run these workflows on their desktop or laptop, while large data resources might be generated on commercial clouds, which provide quick turnaround time to single-core workloads. There are applications, such as those in earthquake science, that need HPC resources to execute parallel jobs within a workflow.

Original language	English
Article number	7373501
Pages (from-to)	70-76
Number of pages	7
Journal	IEEE Internet Computing
Volume	20
Issue number	1
DOIs	https://doi.org/10.1109/MIC.2016.15
State	Published - 2016
Externally published	Yes

Funding

This work was supported by the US National Science Foundation (under grants ACI-1148515, ACI 1245926, and FutureGrid 0910812) and the National Institutes of Health under the NHGRI GS-IT program, grant 1U01 HG006531-01.

Access to Document

10.1109/MIC.2016.15

Cite this

@article{0948bc5637844da6959d4e3b2806ab99,

title = "Pegasus in the cloud: Science automation through workflow technologies",

abstract = "The Pegasus Workflow Management System maps abstract, resource-independent workflow descriptions onto distributed computing resources. Pegasus workflows are portable across different infrastructures, optimizable for performance and efficiency, and automatically map to many different storage systems and data flows. Individual researchers can use astronomy workflows, such as science-grade image mosaicking, to find new celestial objects, or collaborate to generate large data resources that are useful to the entire astronomy community. Individual researchers might run these workflows on their desktop or laptop, while large data resources might be generated on commercial clouds, which provide quick turnaround time to single-core workloads. There are applications, such as those in earthquake science, that need HPC resources to execute parallel jobs within a workflow.",

author = "Ewa Deelman and Karan Vahi and Mats Rynge and Gideon Juve and Rajiv Mayani and \{Da Silva\}, \{Rafael Ferreira\}",

year = "2016",

doi = "10.1109/MIC.2016.15",

language = "English",

volume = "20",

pages = "70--76",

journal = "IEEE Internet Computing",

issn = "1089-7801",

publisher = "Institute of Electrical and Electronics Engineers",

number = "1",

}

TY - JOUR

T1 - Pegasus in the cloud

T2 - Science automation through workflow technologies

AU - Deelman, Ewa

AU - Vahi, Karan

AU - Rynge, Mats

AU - Juve, Gideon

AU - Mayani, Rajiv

AU - Da Silva, Rafael Ferreira

PY - 2016

Y1 - 2016

N2 - The Pegasus Workflow Management System maps abstract, resource-independent workflow descriptions onto distributed computing resources. Pegasus workflows are portable across different infrastructures, optimizable for performance and efficiency, and automatically map to many different storage systems and data flows. Individual researchers can use astronomy workflows, such as science-grade image mosaicking, to find new celestial objects, or collaborate to generate large data resources that are useful to the entire astronomy community. Individual researchers might run these workflows on their desktop or laptop, while large data resources might be generated on commercial clouds, which provide quick turnaround time to single-core workloads. There are applications, such as those in earthquake science, that need HPC resources to execute parallel jobs within a workflow.

AB - The Pegasus Workflow Management System maps abstract, resource-independent workflow descriptions onto distributed computing resources. Pegasus workflows are portable across different infrastructures, optimizable for performance and efficiency, and automatically map to many different storage systems and data flows. Individual researchers can use astronomy workflows, such as science-grade image mosaicking, to find new celestial objects, or collaborate to generate large data resources that are useful to the entire astronomy community. Individual researchers might run these workflows on their desktop or laptop, while large data resources might be generated on commercial clouds, which provide quick turnaround time to single-core workloads. There are applications, such as those in earthquake science, that need HPC resources to execute parallel jobs within a workflow.

UR - https://www.scopus.com/pages/publications/84971455161

U2 - 10.1109/MIC.2016.15

DO - 10.1109/MIC.2016.15

M3 - Article

AN - SCOPUS:84971455161

SN - 1089-7801

VL - 20

SP - 70

EP - 76

JO - IEEE Internet Computing

JF - IEEE Internet Computing

IS - 1

M1 - 7373501

ER -

Pegasus in the cloud: Science automation through workflow technologies

Abstract

Funding

Access to Document

Other files and links

Fingerprint

Cite this