Enabling modern data discovery for atmospheric measurements

Kavya Guntupally; Kyle Dumas; Giri Prakash; Ranjeet Devarakonda; Wade Darnell; Maggie Davis; Richard Cederwall

doi:10.1007/s12145-021-00635-0

Enabling modern data discovery for atmospheric measurements

Kavya Guntupally, Kyle Dumas, Giri Prakash, Ranjeet Devarakonda, Wade Darnell, Maggie Davis, Richard Cederwall

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.

Original language	English
Pages (from-to)	1487-1502
Number of pages	16
Journal	Earth Science Informatics
Volume	14
Issue number	3
DOIs	https://doi.org/10.1007/s12145-021-00635-0
State	Published - Sep 2021

Funding

This research was supported by the Atmospheric Radiation Measurement (ARM) user facility, a U.S. Department of Energy (DOE) Office of Science user facility managed by the Office of Biological and Environmental Research Program. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the US Department of Energy. The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (https://www.energy.gov/downloads/doe-public-access-plan). This research was supported by the Atmospheric Radiation Measurement (ARM) user facility, a U.S. Department of Energy (DOE) Office of Science user facility managed by the Office of Biological and Environmental Research Program. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the US Department of Energy. The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan ( https://www.energy.gov/downloads/doe-public-access-plan ).

Funders	Funder number
DOE Public Access Plan
U.S. Department of Energy
Office of Science
Biological and Environmental Research	DE-AC05-00OR22725

Keywords

ARM data center
Data archive
Data search
FAIR data
Metadata
Metadata management

Access to Document

10.1007/s12145-021-00635-0

Cite this

@article{6f5f31cdc3c4460a98d665a7e86d49b4,

title = "Enabling modern data discovery for atmospheric measurements",

abstract = "The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.",

keywords = "ARM data center, Data archive, Data search, FAIR data, Metadata, Metadata management",

author = "Kavya Guntupally and Kyle Dumas and Giri Prakash and Ranjeet Devarakonda and Wade Darnell and Maggie Davis and Richard Cederwall",

note = "Publisher Copyright: {\textcopyright} 2021, This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply.",

year = "2021",

month = sep,

doi = "10.1007/s12145-021-00635-0",

language = "English",

volume = "14",

pages = "1487--1502",

journal = "Earth Science Informatics",

issn = "1865-0473",

publisher = "Springer",

number = "3",

}

TY - JOUR

T1 - Enabling modern data discovery for atmospheric measurements

AU - Guntupally, Kavya

AU - Dumas, Kyle

AU - Prakash, Giri

AU - Devarakonda, Ranjeet

AU - Darnell, Wade

AU - Davis, Maggie

AU - Cederwall, Richard

PY - 2021/9

Y1 - 2021/9

N2 - The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.

AB - The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.

KW - ARM data center

KW - Data archive

KW - Data search

KW - FAIR data

KW - Metadata

KW - Metadata management

UR - http://www.scopus.com/inward/record.url?scp=85108189462&partnerID=8YFLogxK

U2 - 10.1007/s12145-021-00635-0

DO - 10.1007/s12145-021-00635-0

M3 - Article

AN - SCOPUS:85108189462

SN - 1865-0473

VL - 14

SP - 1487

EP - 1502

JO - Earth Science Informatics

JF - Earth Science Informatics

IS - 3

ER -

Enabling modern data discovery for atmospheric measurements

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this