BeeFlow: A workflow management system for in situ processing across HPC and cloud systems

Jieyang Chen, Qiang Guan, Zhao Zhang, Xin Liang, Louis Vernon, Allen McPherson, Li Ta Lo, Patricia Grubel, Tim Randles, Zizhong Chen, James Ahrens

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

13 Scopus citations

Abstract

In this paper, we propose BeeFlow - an in situ analysis enabled workflow management system across multiple platforms using Docker containers. BeeFlow can support both traditional workflows as well as workflows with in situ analysis. BeeFlow leverages Docker containers to provide a portable, flexible, and reproducible workflow management system across HPC and cloud platforms. We showcase how current in situ visualization workflows can apply BeeFlow with DOE production codes VPIC and Flecsale.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE 38th International Conference on Distributed Computing Systems, ICDCS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1029-1038
Number of pages10
ISBN (Electronic)9781538668719
DOIs
StatePublished - Jul 19 2018
Externally publishedYes
Event38th IEEE International Conference on Distributed Computing Systems, ICDCS 2018 - Vienna, Austria
Duration: Jul 2 2018Jul 5 2018

Publication series

NameProceedings - International Conference on Distributed Computing Systems
Volume2018-July

Conference

Conference38th IEEE International Conference on Distributed Computing Systems, ICDCS 2018
Country/TerritoryAustria
CityVienna
Period07/2/1807/5/18

Keywords

  • Cloud computing
  • Container
  • High performance computing
  • Scientific workflow

Fingerprint

Dive into the research topics of 'BeeFlow: A workflow management system for in situ processing across HPC and cloud systems'. Together they form a unique fingerprint.

Cite this