TY - GEN
T1 - GoldRush
T2 - 2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013
AU - Zheng, Fang
AU - Yu, Hongfeng
AU - Hantas, Can
AU - Wolf, Matthew
AU - Eisenhauer, Greg
AU - Schwan, Karsten
AU - Abbasi, Hasan
AU - Klasky, Scott
PY - 2013
Y1 - 2013
N2 - Severe I/O bottlenecks on High End Computing platforms call for running data analytics in situ. Demonstrating that there exist considerable resources in compute nodes un-used by typical high end scientific simulations, we leverage this fact by creating an agile runtime, termed GoldRush, that can harvest those otherwise wasted, idle resources to efficiently run in situ data analytics. GoldRush uses fine-grained scheduling to "steal" idle resources, in ways that minimize interference between the simulation and in situ analytics. This involves recognizing the potential causes of on-node resource contention and then using scheduling methods that prevent them. Experiments with representative science applications at large scales show that resources harvested on compute nodes can be leveraged to perform useful analytics, significantly improving resource efficiency, reducing data movement costs incurred by alternate solutions, and posing negligible impact on scientific simulations.
AB - Severe I/O bottlenecks on High End Computing platforms call for running data analytics in situ. Demonstrating that there exist considerable resources in compute nodes un-used by typical high end scientific simulations, we leverage this fact by creating an agile runtime, termed GoldRush, that can harvest those otherwise wasted, idle resources to efficiently run in situ data analytics. GoldRush uses fine-grained scheduling to "steal" idle resources, in ways that minimize interference between the simulation and in situ analytics. This involves recognizing the potential causes of on-node resource contention and then using scheduling methods that prevent them. Experiments with representative science applications at large scales show that resources harvested on compute nodes can be leveraged to perform useful analytics, significantly improving resource efficiency, reducing data movement costs incurred by alternate solutions, and posing negligible impact on scientific simulations.
UR - http://www.scopus.com/inward/record.url?scp=84899670941&partnerID=8YFLogxK
U2 - 10.1145/2503210.2503279
DO - 10.1145/2503210.2503279
M3 - Conference contribution
AN - SCOPUS:84899670941
SN - 9781450323789
T3 - International Conference for High Performance Computing, Networking, Storage and Analysis, SC
BT - Proceedings of SC 2013
PB - IEEE Computer Society
Y2 - 17 November 2013 through 22 November 2013
ER -