GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms

Evan Hataishi; Pierre François Dutot; Rafael Ferreira da Silva; Henri Casanova

doi:10.1007/978-3-030-88224-2_11

GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms

Evan Hataishi
, Pierre François Dutot
, Rafael Ferreira da Silva
, Henri Casanova

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Many scientific workflows have computational demands that require the use of compute platforms managed by batch schedulers, which are unfortunately poorly suited to these applications. This work proposes GLUME, a strategy for partitioning a workflow into batch jobs. The novelty is that these jobs are explicitly constructed to minimize overall workflow execution time. Experimental evaluation via simulation of production batch workloads and workflows shows that our heuristic is more effective than previously proposed strategies when executing workflows with moderate to high computational demand.

Original language	English
Title of host publication	Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers
Editors	Dalibor Klusáček, Walfredo Cirne, Gonzalo P. Rodrigo
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	210-230
Number of pages	21
ISBN (Print)	9783030882235
DOIs	https://doi.org/10.1007/978-3-030-88224-2_11
State	Published - 2021
Externally published	Yes
Event	24th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2021 - Virtual, Online Duration: May 21 2021 → May 21 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12985 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2021
City	Virtual, Online
Period	05/21/21 → 05/21/21

Funding

Acknowledgments. This work is partially funded by NSF contracts #1923539 and #1923621: “CyberTraining: Implementation: Small: Integrating core CI literacy and skills into university curricula via simulation-driven activities”, and the H2020-EU.2.1.1.2. project REGALE (grant ID: 956560). This work is partially funded by NSF contracts #1923539 and #1923621: ?CyberTraining: Implementation: Small: Integrating core CI literacy and skills into university curricula via simulation-driven activities?, and the H2020-EU.2.1.1.2. project REGALE (grant ID: 956560).

Keywords

Batch scheduling
Task clustering
Workflows

Access to Document

10.1007/978-3-030-88224-2_11

Cite this

Hataishi, E., Dutot, P. F., Silva, R. F. D., & Casanova, H. (2021). GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms. In D. Klusáček, W. Cirne, & G. P. Rodrigo (Eds.), Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers (pp. 210-230). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12985 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-88224-2_11

Hataishi, Evan ; Dutot, Pierre François ; Silva, Rafael Ferreira da et al. / GLUME : A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms. Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers. editor / Dalibor Klusáček ; Walfredo Cirne ; Gonzalo P. Rodrigo. Springer Science and Business Media Deutschland GmbH, 2021. pp. 210-230 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ecd111f6c4224f74b52bdc82f55e8a19,

title = "GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms",

abstract = "Many scientific workflows have computational demands that require the use of compute platforms managed by batch schedulers, which are unfortunately poorly suited to these applications. This work proposes GLUME, a strategy for partitioning a workflow into batch jobs. The novelty is that these jobs are explicitly constructed to minimize overall workflow execution time. Experimental evaluation via simulation of production batch workloads and workflows shows that our heuristic is more effective than previously proposed strategies when executing workflows with moderate to high computational demand.",

keywords = "Batch scheduling, Task clustering, Workflows",

author = "Evan Hataishi and Dutot, \{Pierre Fran{\c c}ois\} and Silva, \{Rafael Ferreira da\} and Henri Casanova",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 24th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2021 ; Conference date: 21-05-2021 Through 21-05-2021",

year = "2021",

doi = "10.1007/978-3-030-88224-2\_11",

language = "English",

isbn = "9783030882235",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "210--230",

editor = "Dalibor Klus{\'a}{\v c}ek and Walfredo Cirne and Rodrigo, \{Gonzalo P.\}",

booktitle = "Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers",

}

Hataishi, E, Dutot, PF, Silva, RFD & Casanova, H 2021, GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms. in D Klusáček, W Cirne & GP Rodrigo (eds), Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12985 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 210-230, 24th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2021, Virtual, Online, 05/21/21. https://doi.org/10.1007/978-3-030-88224-2_11

GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms. / Hataishi, Evan; Dutot, Pierre François; Silva, Rafael Ferreira da et al.
Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers. ed. / Dalibor Klusáček; Walfredo Cirne; Gonzalo P. Rodrigo. Springer Science and Business Media Deutschland GmbH, 2021. p. 210-230 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12985 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - GLUME

T2 - 24th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2021

AU - Hataishi, Evan

AU - Dutot, Pierre François

AU - Silva, Rafael Ferreira da

AU - Casanova, Henri

PY - 2021

Y1 - 2021

N2 - Many scientific workflows have computational demands that require the use of compute platforms managed by batch schedulers, which are unfortunately poorly suited to these applications. This work proposes GLUME, a strategy for partitioning a workflow into batch jobs. The novelty is that these jobs are explicitly constructed to minimize overall workflow execution time. Experimental evaluation via simulation of production batch workloads and workflows shows that our heuristic is more effective than previously proposed strategies when executing workflows with moderate to high computational demand.

AB - Many scientific workflows have computational demands that require the use of compute platforms managed by batch schedulers, which are unfortunately poorly suited to these applications. This work proposes GLUME, a strategy for partitioning a workflow into batch jobs. The novelty is that these jobs are explicitly constructed to minimize overall workflow execution time. Experimental evaluation via simulation of production batch workloads and workflows shows that our heuristic is more effective than previously proposed strategies when executing workflows with moderate to high computational demand.

KW - Batch scheduling

KW - Task clustering

KW - Workflows

UR - https://www.scopus.com/pages/publications/85117481680

U2 - 10.1007/978-3-030-88224-2_11

DO - 10.1007/978-3-030-88224-2_11

M3 - Conference contribution

AN - SCOPUS:85117481680

SN - 9783030882235

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 210

EP - 230

BT - Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers

A2 - Klusáček, Dalibor

A2 - Cirne, Walfredo

A2 - Rodrigo, Gonzalo P.

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 21 May 2021 through 21 May 2021

ER -

Hataishi E, Dutot PF, Silva RFD, Casanova H. GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms. In Klusáček D, Cirne W, Rodrigo GP, editors, Job Scheduling Strategies for Parallel Processing - 24th International Workshop, JSSPP 2021, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2021. p. 210-230. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-88224-2_11

GLUME: A Strategy for Reducing Workflow Execution Times on Batch-Scheduled Platforms

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this