TY - GEN
T1 - In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows
AU - Sun, Qian
AU - Romanus, Melissa
AU - Jin, Tong
AU - Yu, Hongfeng
AU - Bremer, Peer Timo
AU - Petruzza, Steve
AU - Klasky, Scott
AU - Parashar, Manish
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2016/11/13
Y1 - 2016/11/13
N2 - Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.
AB - Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.
KW - Couplings
KW - Data storage systems
KW - Runtime
UR - https://www.scopus.com/pages/publications/85013982634
U2 - 10.1109/ESPM2.2016.006
DO - 10.1109/ESPM2.2016.006
M3 - Conference contribution
AN - SCOPUS:85013982634
T3 - Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis
SP - 2
EP - 9
BT - Proceedings of ESPM2 2016
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016 Held in conjunction with The International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016
Y2 - 13 November 2016 through 18 November 2016
ER -