DART: A substrate for high speed asynchronous data IO

Ciprian Docan; Manish Parashar; Scott Klasky

doi:10.1145/1383422.1383454

DART: A substrate for high speed asynchronous data IO

Ciprian Docan, Manish Parashar, Scott Klasky

Workflow Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

24 Scopus citations

Abstract

Large scale simulations of complex physics phenomena have long run times and generate massive amounts of data. Saving this data to external storage systems or transferring it to remote locations for analysis is a costly operation that quickly becomes a performance bottleneck. In this paper, we present DART (Decoupled and Asynchronous Remote Transfers), an efficient data transfer substrate that effectively minimizes the data I/O overhead on the running simulations. DART is a thin software layer built on RDMA technology to enable fast, low-overhead and asynchronous access to data from a running simulation, and support high-throughput, low-latency data transfers.

Original language	English
Title of host publication	Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08
Pages	219-220
Number of pages	2
DOIs	https://doi.org/10.1145/1383422.1383454
State	Published - 2008
Event	17th International Symposium on High Performance Distributed Computing 2008, HPDC'08 - Boston, MA, United States Duration: Jun 23 2008 → Jun 27 2008

Publication series

Name	Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08

Conference

Conference	17th International Symposium on High Performance Distributed Computing 2008, HPDC'08
Country/Territory	United States
City	Boston, MA
Period	06/23/08 → 06/27/08

Keywords

Experimentation

Access to Document

10.1145/1383422.1383454

Cite this

@inproceedings{ef7e5986b9a4466a9efd0812ecd1ec6a,

title = "DART: A substrate for high speed asynchronous data IO",

abstract = "Large scale simulations of complex physics phenomena have long run times and generate massive amounts of data. Saving this data to external storage systems or transferring it to remote locations for analysis is a costly operation that quickly becomes a performance bottleneck. In this paper, we present DART (Decoupled and Asynchronous Remote Transfers), an efficient data transfer substrate that effectively minimizes the data I/O overhead on the running simulations. DART is a thin software layer built on RDMA technology to enable fast, low-overhead and asynchronous access to data from a running simulation, and support high-throughput, low-latency data transfers.",

keywords = "Experimentation",

author = "Ciprian Docan and Manish Parashar and Scott Klasky",

year = "2008",

doi = "10.1145/1383422.1383454",

language = "English",

isbn = "9781595939975",

series = "Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08",

pages = "219--220",

booktitle = "Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08",

note = "17th International Symposium on High Performance Distributed Computing 2008, HPDC'08 ; Conference date: 23-06-2008 Through 27-06-2008",

}

Docan, C, Parashar, M & Klasky, S 2008, DART: A substrate for high speed asynchronous data IO. in Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08. Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08, pp. 219-220, 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08, Boston, MA, United States, 06/23/08. https://doi.org/10.1145/1383422.1383454

DART: A substrate for high speed asynchronous data IO. / Docan, Ciprian; Parashar, Manish; Klasky, Scott.
Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08. 2008. p. 219-220 (Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - DART

T2 - 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08

AU - Docan, Ciprian

AU - Parashar, Manish

AU - Klasky, Scott

PY - 2008

Y1 - 2008

N2 - Large scale simulations of complex physics phenomena have long run times and generate massive amounts of data. Saving this data to external storage systems or transferring it to remote locations for analysis is a costly operation that quickly becomes a performance bottleneck. In this paper, we present DART (Decoupled and Asynchronous Remote Transfers), an efficient data transfer substrate that effectively minimizes the data I/O overhead on the running simulations. DART is a thin software layer built on RDMA technology to enable fast, low-overhead and asynchronous access to data from a running simulation, and support high-throughput, low-latency data transfers.

AB - Large scale simulations of complex physics phenomena have long run times and generate massive amounts of data. Saving this data to external storage systems or transferring it to remote locations for analysis is a costly operation that quickly becomes a performance bottleneck. In this paper, we present DART (Decoupled and Asynchronous Remote Transfers), an efficient data transfer substrate that effectively minimizes the data I/O overhead on the running simulations. DART is a thin software layer built on RDMA technology to enable fast, low-overhead and asynchronous access to data from a running simulation, and support high-throughput, low-latency data transfers.

KW - Experimentation

UR - http://www.scopus.com/inward/record.url?scp=57349125645&partnerID=8YFLogxK

U2 - 10.1145/1383422.1383454

DO - 10.1145/1383422.1383454

M3 - Conference contribution

AN - SCOPUS:57349125645

SN - 9781595939975

T3 - Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08

SP - 219

EP - 220

BT - Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08

Y2 - 23 June 2008 through 27 June 2008

ER -

DART: A substrate for high speed asynchronous data IO

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this