Workflow Submit Nodes as a Service on Leadership Class Systems

George Papadimitriou; Karan Vahi; Jason Kincl; Valentine Anantharaj; Ewa Deelman; Jack Wells

doi:10.1145/3311790.3396671

Workflow Submit Nodes as a Service on Leadership Class Systems

George Papadimitriou, Karan Vahi, Jason Kincl, Valentine Anantharaj, Ewa Deelman, Jack Wells

Data Lifecycle Technologies

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

DOE scientists, today, have access to high performance computing (HPC) facilities with very powerful systems that enable them to execute their computations faster, more efficiently, and at greater scales than ever before. To further their knowledge and produce new discoveries, scientists rely on workflows - sometimes very complex - that provide them with an easy way to automate, reproduce and verify their computations. However, historically, creating workflow submission environments in large HPC facilities has been cumbersome, requires expertise and many man-hours of effort due to the peculiarities, policies, and the restrictions that these systems present. In this paper we discuss the approach a large DOE facility (OLCF) is taking in order to provide containers as a service to its users. This capability is used to create Pegasus workflow management system submit nodes as a service (WSaaS) at the Oak Ridge Leadership Computing Facilities (OLCF), targeting the Summit supercomputer. This deployment builds upon the Kubernetes/Openshift cluster (Slate) that exists within OLCF's DMZ and its automation triggers. Additionally, we evaluate our approach's overhead and effort to deploy the solution as compared to previous solutions, such as setting up a Pegasus submission environment on OLCF's login nodes or submitting jobs remotely via the rvGAHP.

Original language	English
Title of host publication	PEARC 2020 - Practice and Experience in Advanced Research Computing 2020
Subtitle of host publication	Catch the Wave
Publisher	Association for Computing Machinery
Pages	56-63
Number of pages	8
ISBN (Electronic)	9781450366892
DOIs	https://doi.org/10.1145/3311790.3396671
State	Published - Jul 26 2020
Event	2020 Conference on Practice and Experience in Advanced Research Computing: Catch the Wave, PEARC 2020 - Virtual, Online, United States Duration: Jul 27 2020 → Jul 31 2020

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	2020 Conference on Practice and Experience in Advanced Research Computing: Catch the Wave, PEARC 2020
Country/Territory	United States
City	Virtual, Online
Period	07/27/20 → 07/31/20

Funding

This work was funded by DOE contract number #DESC0012636, “Panorama—Predictive Modeling and Diagnostic Monitoring of Extreme Science Workflows”, and by the U.S. Department of Energy, Office of Science under contract DE-AC02-06CH11357. Also, this research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725. Finally, we would like to thank Scott Callaghan from SCEC for his help in debugging the rvGAHP deployment on Summit’s login nodes and OLCF’s data transfer nodes (DTNs). This work was funded by DOE contract number #DESC0012636, ?Panorama?Predictive Modeling and Diagnostic Monitoring of Extreme Science Workflows?, and by the U.S. Department of Energy, Office of Science under contract DE-AC02-06CH11357. Also, this research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725. Finally, we would like to thank Scott Callaghan from SCEC for his help in debugging the rvGAHP deployment on Summit?s login nodes and OLCF?s data transfer nodes (DTNs).

Keywords

Kubernetes
Pegasus
Scientific Workflows
Summit Supercomputer

Access to Document

10.1145/3311790.3396671

Cite this

Papadimitriou, G., Vahi, K., Kincl, J., Anantharaj, V., Deelman, E., & Wells, J. (2020). Workflow Submit Nodes as a Service on Leadership Class Systems. In PEARC 2020 - Practice and Experience in Advanced Research Computing 2020: Catch the Wave (pp. 56-63). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3311790.3396671

@inproceedings{82b60f51b3f049bf86d406c82096f33c,

title = "Workflow Submit Nodes as a Service on Leadership Class Systems",

abstract = "DOE scientists, today, have access to high performance computing (HPC) facilities with very powerful systems that enable them to execute their computations faster, more efficiently, and at greater scales than ever before. To further their knowledge and produce new discoveries, scientists rely on workflows - sometimes very complex - that provide them with an easy way to automate, reproduce and verify their computations. However, historically, creating workflow submission environments in large HPC facilities has been cumbersome, requires expertise and many man-hours of effort due to the peculiarities, policies, and the restrictions that these systems present. In this paper we discuss the approach a large DOE facility (OLCF) is taking in order to provide containers as a service to its users. This capability is used to create Pegasus workflow management system submit nodes as a service (WSaaS) at the Oak Ridge Leadership Computing Facilities (OLCF), targeting the Summit supercomputer. This deployment builds upon the Kubernetes/Openshift cluster (Slate) that exists within OLCF's DMZ and its automation triggers. Additionally, we evaluate our approach's overhead and effort to deploy the solution as compared to previous solutions, such as setting up a Pegasus submission environment on OLCF's login nodes or submitting jobs remotely via the rvGAHP.",

keywords = "Kubernetes, Pegasus, Scientific Workflows, Summit Supercomputer",

author = "George Papadimitriou and Karan Vahi and Jason Kincl and Valentine Anantharaj and Ewa Deelman and Jack Wells",

note = "Publisher Copyright: {\textcopyright} 2020 ACM.; 2020 Conference on Practice and Experience in Advanced Research Computing: Catch the Wave, PEARC 2020 ; Conference date: 27-07-2020 Through 31-07-2020",

year = "2020",

month = jul,

day = "26",

doi = "10.1145/3311790.3396671",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "56--63",

booktitle = "PEARC 2020 - Practice and Experience in Advanced Research Computing 2020",

}

Papadimitriou, G, Vahi, K, Kincl, J, Anantharaj, V, Deelman, E & Wells, J 2020, Workflow Submit Nodes as a Service on Leadership Class Systems. in PEARC 2020 - Practice and Experience in Advanced Research Computing 2020: Catch the Wave. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 56-63, 2020 Conference on Practice and Experience in Advanced Research Computing: Catch the Wave, PEARC 2020, Virtual, Online, United States, 07/27/20. https://doi.org/10.1145/3311790.3396671

Workflow Submit Nodes as a Service on Leadership Class Systems. / Papadimitriou, George; Vahi, Karan; Kincl, Jason et al.
PEARC 2020 - Practice and Experience in Advanced Research Computing 2020: Catch the Wave. Association for Computing Machinery, 2020. p. 56-63 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Workflow Submit Nodes as a Service on Leadership Class Systems

AU - Papadimitriou, George

AU - Vahi, Karan

AU - Kincl, Jason

AU - Anantharaj, Valentine

AU - Deelman, Ewa

AU - Wells, Jack

PY - 2020/7/26

Y1 - 2020/7/26

N2 - DOE scientists, today, have access to high performance computing (HPC) facilities with very powerful systems that enable them to execute their computations faster, more efficiently, and at greater scales than ever before. To further their knowledge and produce new discoveries, scientists rely on workflows - sometimes very complex - that provide them with an easy way to automate, reproduce and verify their computations. However, historically, creating workflow submission environments in large HPC facilities has been cumbersome, requires expertise and many man-hours of effort due to the peculiarities, policies, and the restrictions that these systems present. In this paper we discuss the approach a large DOE facility (OLCF) is taking in order to provide containers as a service to its users. This capability is used to create Pegasus workflow management system submit nodes as a service (WSaaS) at the Oak Ridge Leadership Computing Facilities (OLCF), targeting the Summit supercomputer. This deployment builds upon the Kubernetes/Openshift cluster (Slate) that exists within OLCF's DMZ and its automation triggers. Additionally, we evaluate our approach's overhead and effort to deploy the solution as compared to previous solutions, such as setting up a Pegasus submission environment on OLCF's login nodes or submitting jobs remotely via the rvGAHP.

AB - DOE scientists, today, have access to high performance computing (HPC) facilities with very powerful systems that enable them to execute their computations faster, more efficiently, and at greater scales than ever before. To further their knowledge and produce new discoveries, scientists rely on workflows - sometimes very complex - that provide them with an easy way to automate, reproduce and verify their computations. However, historically, creating workflow submission environments in large HPC facilities has been cumbersome, requires expertise and many man-hours of effort due to the peculiarities, policies, and the restrictions that these systems present. In this paper we discuss the approach a large DOE facility (OLCF) is taking in order to provide containers as a service to its users. This capability is used to create Pegasus workflow management system submit nodes as a service (WSaaS) at the Oak Ridge Leadership Computing Facilities (OLCF), targeting the Summit supercomputer. This deployment builds upon the Kubernetes/Openshift cluster (Slate) that exists within OLCF's DMZ and its automation triggers. Additionally, we evaluate our approach's overhead and effort to deploy the solution as compared to previous solutions, such as setting up a Pegasus submission environment on OLCF's login nodes or submitting jobs remotely via the rvGAHP.

KW - Kubernetes

KW - Pegasus

KW - Scientific Workflows

KW - Summit Supercomputer

UR - http://www.scopus.com/inward/record.url?scp=85089277256&partnerID=8YFLogxK

U2 - 10.1145/3311790.3396671

DO - 10.1145/3311790.3396671

M3 - Conference contribution

AN - SCOPUS:85089277256

T3 - ACM International Conference Proceeding Series

SP - 56

EP - 63

BT - PEARC 2020 - Practice and Experience in Advanced Research Computing 2020

PB - Association for Computing Machinery

T2 - 2020 Conference on Practice and Experience in Advanced Research Computing: Catch the Wave, PEARC 2020

Y2 - 27 July 2020 through 31 July 2020

ER -

Workflow Submit Nodes as a Service on Leadership Class Systems

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this