DataFed: Towards reproducible research via federated data management

Dale Stansberry; Suhas Somnath; Jessica Breet; Gregory Shutt; Mallikarjun Shankar

doi:10.1109/CSCI49370.2019.00245

DataFed: Towards reproducible research via federated data management

Dale Stansberry, Suhas Somnath, Jessica Breet, Gregory Shutt, Mallikarjun Shankar

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Scopus citations

Abstract

The increasingly collaborative, globalized nature of scientific research combined with the need to share data and the explosion in data volumes present an urgent need for a scientific data management system (SDMS). An SDMS presents a logical and holistic view of data that greatly simplifies and empowers data organization, curation, searching, sharing, dissemination, etc. We present DataFed - a lightweight, distributed SDMS that spans a federation of storage systems within a loosely-coupled network of scientific facilities. Unlike existing SDMS offerings, DataFed uses high-performance and scalable user management and data transfer technologies that simplify deployment, maintenance, and expansion of DataFed. DataFed provides web-based and command-line interfaces to manage data and integrate with complex scientific workflows. DataFed represents a step towards reproducible scientific research by enabling reliable staging of the correct data at the desired environment.

Original language	English
Title of host publication	Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1312-1317
Number of pages	6
ISBN (Electronic)	9781728155845
DOIs	https://doi.org/10.1109/CSCI49370.2019.00245
State	Published - Dec 2019
Event	6th Annual International Conference on Computational Science and Computational Intelligence, CSCI 2019 - Las Vegas, United States Duration: Dec 5 2019 → Dec 7 2019

Publication series

Name	Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019

Conference

Conference	6th Annual International Conference on Computational Science and Computational Intelligence, CSCI 2019
Country/Territory	United States
City	Las Vegas
Period	12/5/19 → 12/7/19

Funding

This research used resources of the Oak Ridge Leadership Computing Facility (OLCF) and of the Compute and Data Environment for Science (CADES) at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Keywords

Cross-facility
FAIR data principles
Federated identity management
Globus
Scientific data management system

Access to Document

10.1109/CSCI49370.2019.00245

Cite this

Stansberry, D., Somnath, S., Breet, J., Shutt, G., & Shankar, M. (2019). DataFed: Towards reproducible research via federated data management. In Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019 (pp. 1312-1317). Article 9071425 (Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CSCI49370.2019.00245

Stansberry, Dale ; Somnath, Suhas ; Breet, Jessica et al. / DataFed : Towards reproducible research via federated data management. Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1312-1317 (Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019).

@inproceedings{a7accd530c304e46959a834e57f9fb9d,

title = "DataFed: Towards reproducible research via federated data management",

abstract = "The increasingly collaborative, globalized nature of scientific research combined with the need to share data and the explosion in data volumes present an urgent need for a scientific data management system (SDMS). An SDMS presents a logical and holistic view of data that greatly simplifies and empowers data organization, curation, searching, sharing, dissemination, etc. We present DataFed - a lightweight, distributed SDMS that spans a federation of storage systems within a loosely-coupled network of scientific facilities. Unlike existing SDMS offerings, DataFed uses high-performance and scalable user management and data transfer technologies that simplify deployment, maintenance, and expansion of DataFed. DataFed provides web-based and command-line interfaces to manage data and integrate with complex scientific workflows. DataFed represents a step towards reproducible scientific research by enabling reliable staging of the correct data at the desired environment.",

keywords = "Cross-facility, FAIR data principles, Federated identity management, Globus, Scientific data management system",

author = "Dale Stansberry and Suhas Somnath and Jessica Breet and Gregory Shutt and Mallikarjun Shankar",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 6th Annual International Conference on Computational Science and Computational Intelligence, CSCI 2019 ; Conference date: 05-12-2019 Through 07-12-2019",

year = "2019",

month = dec,

doi = "10.1109/CSCI49370.2019.00245",

language = "English",

series = "Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1312--1317",

booktitle = "Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019",

}

Stansberry, D, Somnath, S, Breet, J, Shutt, G & Shankar, M 2019, DataFed: Towards reproducible research via federated data management. in Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019., 9071425, Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019, Institute of Electrical and Electronics Engineers Inc., pp. 1312-1317, 6th Annual International Conference on Computational Science and Computational Intelligence, CSCI 2019, Las Vegas, United States, 12/5/19. https://doi.org/10.1109/CSCI49370.2019.00245

DataFed: Towards reproducible research via federated data management. / Stansberry, Dale; Somnath, Suhas; Breet, Jessica et al.
Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1312-1317 9071425 (Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - DataFed

T2 - 6th Annual International Conference on Computational Science and Computational Intelligence, CSCI 2019

AU - Stansberry, Dale

AU - Somnath, Suhas

AU - Breet, Jessica

AU - Shutt, Gregory

AU - Shankar, Mallikarjun

PY - 2019/12

Y1 - 2019/12

N2 - The increasingly collaborative, globalized nature of scientific research combined with the need to share data and the explosion in data volumes present an urgent need for a scientific data management system (SDMS). An SDMS presents a logical and holistic view of data that greatly simplifies and empowers data organization, curation, searching, sharing, dissemination, etc. We present DataFed - a lightweight, distributed SDMS that spans a federation of storage systems within a loosely-coupled network of scientific facilities. Unlike existing SDMS offerings, DataFed uses high-performance and scalable user management and data transfer technologies that simplify deployment, maintenance, and expansion of DataFed. DataFed provides web-based and command-line interfaces to manage data and integrate with complex scientific workflows. DataFed represents a step towards reproducible scientific research by enabling reliable staging of the correct data at the desired environment.

AB - The increasingly collaborative, globalized nature of scientific research combined with the need to share data and the explosion in data volumes present an urgent need for a scientific data management system (SDMS). An SDMS presents a logical and holistic view of data that greatly simplifies and empowers data organization, curation, searching, sharing, dissemination, etc. We present DataFed - a lightweight, distributed SDMS that spans a federation of storage systems within a loosely-coupled network of scientific facilities. Unlike existing SDMS offerings, DataFed uses high-performance and scalable user management and data transfer technologies that simplify deployment, maintenance, and expansion of DataFed. DataFed provides web-based and command-line interfaces to manage data and integrate with complex scientific workflows. DataFed represents a step towards reproducible scientific research by enabling reliable staging of the correct data at the desired environment.

KW - Cross-facility

KW - FAIR data principles

KW - Federated identity management

KW - Globus

KW - Scientific data management system

UR - http://www.scopus.com/inward/record.url?scp=85084728594&partnerID=8YFLogxK

U2 - 10.1109/CSCI49370.2019.00245

DO - 10.1109/CSCI49370.2019.00245

M3 - Conference contribution

AN - SCOPUS:85084728594

T3 - Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019

SP - 1312

EP - 1317

BT - Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 5 December 2019 through 7 December 2019

ER -

Stansberry D, Somnath S, Breet J, Shutt G, Shankar M. DataFed: Towards reproducible research via federated data management. In Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1312-1317. 9071425. (Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019). doi: 10.1109/CSCI49370.2019.00245

DataFed: Towards reproducible research via federated data management

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this