Scibox: Online sharing of scientific data via the cloud

Jian Huang; Xuechen Zhang; Greg Eisenhauer; Karsten Schwan; Matthew Wolf; Stephane Ethier; Scott Klasky

doi:10.1109/IPDPS.2014.26

Scibox: Online sharing of scientific data via the cloud

Jian Huang, Xuechen Zhang, Greg Eisenhauer, Karsten Schwan, Matthew Wolf, Stephane Ethier, Scott Klasky

Workflow Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

Collaborative science demands global sharing of scientific data. But it cannot leverage universally accessible cloud-based infrastructures like Drop Box, as those offer limited interfaces and inadequate levels of access bandwidth. We present the Scibox cloud facility for online sharing scientific data. It uses standard cloud storage solutions, but offers a usage model in which high end codes can write/read data to/from the cloud via the APIs they already use for their I/O actions. With Scibox, data upload/download volumes are controlled via Data Reduction-functions stated by end users and applied at the data source, before data is moved, with further gains in efficiency obtained by combining DR-functions to move exactly what is needed by current data consumers. We evaluate Scibox with science applications and their representative data analytics - the GTS fusion and the combustion image processing - demonstrating the potential for ubiquitous data access with substantial reductions in network traffic.

Original language	English
Title of host publication	Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014
Publisher	IEEE Computer Society
Pages	145-154
Number of pages	10
ISBN (Print)	9780769552071
DOIs	https://doi.org/10.1109/IPDPS.2014.26
State	Published - 2014
Event	28th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2014 - Phoenix, AZ, United States Duration: May 19 2014 → May 23 2014

Publication series

Name	Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS
ISSN (Print)	1530-2075
ISSN (Electronic)	2332-1237

Conference

Conference	28th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2014
Country/Territory	United States
City	Phoenix, AZ
Period	05/19/14 → 05/23/14

Keywords

Cloud Storage
Data Sharing
Scientific Data

Access to Document

10.1109/IPDPS.2014.26

Cite this

Huang, J., Zhang, X., Eisenhauer, G., Schwan, K., Wolf, M., Ethier, S., & Klasky, S. (2014). Scibox: Online sharing of scientific data via the cloud. In Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014 (pp. 145-154). Article 6877250 (Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS). IEEE Computer Society. https://doi.org/10.1109/IPDPS.2014.26

@inproceedings{232b24a22f8a4286bd64a5216f786455,

title = "Scibox: Online sharing of scientific data via the cloud",

abstract = "Collaborative science demands global sharing of scientific data. But it cannot leverage universally accessible cloud-based infrastructures like Drop Box, as those offer limited interfaces and inadequate levels of access bandwidth. We present the Scibox cloud facility for online sharing scientific data. It uses standard cloud storage solutions, but offers a usage model in which high end codes can write/read data to/from the cloud via the APIs they already use for their I/O actions. With Scibox, data upload/download volumes are controlled via Data Reduction-functions stated by end users and applied at the data source, before data is moved, with further gains in efficiency obtained by combining DR-functions to move exactly what is needed by current data consumers. We evaluate Scibox with science applications and their representative data analytics - the GTS fusion and the combustion image processing - demonstrating the potential for ubiquitous data access with substantial reductions in network traffic.",

keywords = "Cloud Storage, Data Sharing, Scientific Data",

author = "Jian Huang and Xuechen Zhang and Greg Eisenhauer and Karsten Schwan and Matthew Wolf and Stephane Ethier and Scott Klasky",

year = "2014",

doi = "10.1109/IPDPS.2014.26",

language = "English",

isbn = "9780769552071",

series = "Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS",

publisher = "IEEE Computer Society",

pages = "145--154",

booktitle = "Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014",

note = "28th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2014 ; Conference date: 19-05-2014 Through 23-05-2014",

}

Huang, J, Zhang, X, Eisenhauer, G, Schwan, K, Wolf, M, Ethier, S & Klasky, S 2014, Scibox: Online sharing of scientific data via the cloud. in Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014., 6877250, Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS, IEEE Computer Society, pp. 145-154, 28th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2014, Phoenix, AZ, United States, 05/19/14. https://doi.org/10.1109/IPDPS.2014.26

Scibox: Online sharing of scientific data via the cloud. / Huang, Jian; Zhang, Xuechen; Eisenhauer, Greg et al.
Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014. IEEE Computer Society, 2014. p. 145-154 6877250 (Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Scibox

T2 - 28th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2014

AU - Huang, Jian

AU - Zhang, Xuechen

AU - Eisenhauer, Greg

AU - Schwan, Karsten

AU - Wolf, Matthew

AU - Ethier, Stephane

AU - Klasky, Scott

PY - 2014

Y1 - 2014

N2 - Collaborative science demands global sharing of scientific data. But it cannot leverage universally accessible cloud-based infrastructures like Drop Box, as those offer limited interfaces and inadequate levels of access bandwidth. We present the Scibox cloud facility for online sharing scientific data. It uses standard cloud storage solutions, but offers a usage model in which high end codes can write/read data to/from the cloud via the APIs they already use for their I/O actions. With Scibox, data upload/download volumes are controlled via Data Reduction-functions stated by end users and applied at the data source, before data is moved, with further gains in efficiency obtained by combining DR-functions to move exactly what is needed by current data consumers. We evaluate Scibox with science applications and their representative data analytics - the GTS fusion and the combustion image processing - demonstrating the potential for ubiquitous data access with substantial reductions in network traffic.

AB - Collaborative science demands global sharing of scientific data. But it cannot leverage universally accessible cloud-based infrastructures like Drop Box, as those offer limited interfaces and inadequate levels of access bandwidth. We present the Scibox cloud facility for online sharing scientific data. It uses standard cloud storage solutions, but offers a usage model in which high end codes can write/read data to/from the cloud via the APIs they already use for their I/O actions. With Scibox, data upload/download volumes are controlled via Data Reduction-functions stated by end users and applied at the data source, before data is moved, with further gains in efficiency obtained by combining DR-functions to move exactly what is needed by current data consumers. We evaluate Scibox with science applications and their representative data analytics - the GTS fusion and the combustion image processing - demonstrating the potential for ubiquitous data access with substantial reductions in network traffic.

KW - Cloud Storage

KW - Data Sharing

KW - Scientific Data

UR - https://www.scopus.com/pages/publications/84906717603

U2 - 10.1109/IPDPS.2014.26

DO - 10.1109/IPDPS.2014.26

M3 - Conference contribution

AN - SCOPUS:84906717603

SN - 9780769552071

T3 - Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS

SP - 145

EP - 154

BT - Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014

PB - IEEE Computer Society

Y2 - 19 May 2014 through 23 May 2014

ER -

Huang J, Zhang X, Eisenhauer G, Schwan K, Wolf M, Ethier S et al. Scibox: Online sharing of scientific data via the cloud. In Proceedings - IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS 2014. IEEE Computer Society. 2014. p. 145-154. 6877250. (Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS). doi: 10.1109/IPDPS.2014.26

Scibox: Online sharing of scientific data via the cloud

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this