In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

Qian Sun; Melissa Romanus; Tong Jin; Hongfeng Yu; Peer Timo Bremer; Steve Petruzza; Scott Klasky; Manish Parashar

doi:10.1109/ESPM2.2016.006

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

Qian Sun, Melissa Romanus, Tong Jin, Hongfeng Yu, Peer Timo Bremer, Steve Petruzza, Scott Klasky, Manish Parashar

Workflow Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

Original language	English
Title of host publication	Proceedings of ESPM2 2016
Subtitle of host publication	2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2-9
Number of pages	8
ISBN (Electronic)	9781509038589
DOIs	https://doi.org/10.1109/ESPM2.2016.006
State	Published - Jan 24 2017
Event	2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016 - Salt Lake City, United States Duration: Nov 18 2016 → …

Publication series

Name	Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016
Country/Territory	United States
City	Salt Lake City
Period	11/18/16 → …

Keywords

Couplings
Data storage systems
Runtime

Access to Document

10.1109/ESPM2.2016.006

Cite this

Sun, Q., Romanus, M., Jin, T., Yu, H., Bremer, P. T., Petruzza, S., Klasky, S., & Parashar, M. (2017). In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. In Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 2-9). Article 7831554 (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESPM2.2016.006

Sun, Qian ; Romanus, Melissa ; Jin, Tong et al. / In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 2-9 (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{b673a1d25346485782186085579eba41,

title = "In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows",

abstract = "Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.",

keywords = "Couplings, Data storage systems, Runtime",

author = "Qian Sun and Melissa Romanus and Tong Jin and Hongfeng Yu and Bremer, {Peer Timo} and Steve Petruzza and Scott Klasky and Manish Parashar",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016 ; Conference date: 18-11-2016",

year = "2017",

month = jan,

day = "24",

doi = "10.1109/ESPM2.2016.006",

language = "English",

series = "Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2--9",

booktitle = "Proceedings of ESPM2 2016",

}

Sun, Q, Romanus, M, Jin, T, Yu, H, Bremer, PT, Petruzza, S, Klasky, S & Parashar, M 2017, In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. in Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis., 7831554, Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 2-9, 2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016, Salt Lake City, United States, 11/18/16. https://doi.org/10.1109/ESPM2.2016.006

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. / Sun, Qian; Romanus, Melissa; Jin, Tong et al.
Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2-9 7831554 (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

AU - Sun, Qian

AU - Romanus, Melissa

AU - Jin, Tong

AU - Yu, Hongfeng

AU - Bremer, Peer Timo

AU - Petruzza, Steve

AU - Klasky, Scott

AU - Parashar, Manish

PY - 2017/1/24

Y1 - 2017/1/24

N2 - Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

AB - Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

KW - Couplings

KW - Data storage systems

KW - Runtime

UR - http://www.scopus.com/inward/record.url?scp=85013982634&partnerID=8YFLogxK

U2 - 10.1109/ESPM2.2016.006

DO - 10.1109/ESPM2.2016.006

M3 - Conference contribution

AN - SCOPUS:85013982634

T3 - Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 2

EP - 9

BT - Proceedings of ESPM2 2016

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016

Y2 - 18 November 2016

ER -

Sun Q, Romanus M, Jin T, Yu H, Bremer PT, Petruzza S et al. In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. In Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2017. p. 2-9. 7831554. (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/ESPM2.2016.006

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this