Next-gen tools for big scientific data: ARM data center example

Ranjeet Devarakonda; Kyle Dumas; Sheman Beus; Everett Rush; Bhargavi Krishna; Rob Records; Giri Prakash

doi:10.1109/BigData.2016.7841078

Next-gen tools for big scientific data: ARM data center example

Ranjeet Devarakonda, Kyle Dumas, Sheman Beus, Everett Rush, Bhargavi Krishna, Rob Records, Giri Prakash

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

The Atmospheric Radiation Measurement (ARM) Climate Research Facility (www.arm.gov) provides atmospheric observations from diverse climatic regimes around the world. Currently, ARM archives over 22 million user assessable data files, primarily stored in NetCDF file format, with total data volumes close to one Petabyte. In this paper, we will discuss how ARM is currently storing, distributing, cataloging and visualizing such large volumes of multi-dimensional climate observations and model data and also describe their future plan.

Original language	English
Title of host publication	Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
Editors	James Joshi, George Karypis, Ling Liu, Xiaohua Tony Hu, Ronay Ak, Yinglong Xia, Weijia Xu, Aki-Hiro Sato, Sudarsan Rachuri, Lyle Ungar, Philip S. Yu, Rama Govindaraju, Toyotaro Suzumura
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3968-3970
Number of pages	3
ISBN (Electronic)	9781467390040
DOIs	https://doi.org/10.1109/BigData.2016.7841078
State	Published - 2016
Event	4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States Duration: Dec 5 2016 → Dec 8 2016

Publication series

Name	Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

Conference

Conference	4th IEEE International Conference on Big Data, Big Data 2016
Country/Territory	United States
City	Washington
Period	12/5/16 → 12/8/16

Keywords

ARM Archive
Big Data Discovery
Metadata Management

Access to Document

10.1109/BigData.2016.7841078

Cite this

Devarakonda, R., Dumas, K., Beus, S., Rush, E., Krishna, B., Records, R., & Prakash, G. (2016). Next-gen tools for big scientific data: ARM data center example. In J. Joshi, G. Karypis, L. Liu, X. T. Hu, R. Ak, Y. Xia, W. Xu, A.-H. Sato, S. Rachuri, L. Ungar, P. S. Yu, R. Govindaraju, & T. Suzumura (Eds.), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 3968-3970). Article 7841078 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7841078

Devarakonda, Ranjeet ; Dumas, Kyle ; Beus, Sheman et al. / Next-gen tools for big scientific data : ARM data center example. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. editor / James Joshi ; George Karypis ; Ling Liu ; Xiaohua Tony Hu ; Ronay Ak ; Yinglong Xia ; Weijia Xu ; Aki-Hiro Sato ; Sudarsan Rachuri ; Lyle Ungar ; Philip S. Yu ; Rama Govindaraju ; Toyotaro Suzumura. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 3968-3970 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

@inproceedings{aca6109183fe421d8f078d611326191f,

title = "Next-gen tools for big scientific data: ARM data center example",

abstract = "The Atmospheric Radiation Measurement (ARM) Climate Research Facility (www.arm.gov) provides atmospheric observations from diverse climatic regimes around the world. Currently, ARM archives over 22 million user assessable data files, primarily stored in NetCDF file format, with total data volumes close to one Petabyte. In this paper, we will discuss how ARM is currently storing, distributing, cataloging and visualizing such large volumes of multi-dimensional climate observations and model data and also describe their future plan.",

keywords = "ARM Archive, Big Data Discovery, Metadata Management",

author = "Ranjeet Devarakonda and Kyle Dumas and Sheman Beus and Everett Rush and Bhargavi Krishna and Rob Records and Giri Prakash",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 4th IEEE International Conference on Big Data, Big Data 2016 ; Conference date: 05-12-2016 Through 08-12-2016",

year = "2016",

doi = "10.1109/BigData.2016.7841078",

language = "English",

series = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3968--3970",

editor = "James Joshi and George Karypis and Ling Liu and Hu, \{Xiaohua Tony\} and Ronay Ak and Yinglong Xia and Weijia Xu and Aki-Hiro Sato and Sudarsan Rachuri and Lyle Ungar and Yu, \{Philip S.\} and Rama Govindaraju and Toyotaro Suzumura",

booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

}

Devarakonda, R, Dumas, K, Beus, S, Rush, E, Krishna, B , Records, R & Prakash, G 2016, Next-gen tools for big scientific data: ARM data center example. in J Joshi, G Karypis, L Liu, XT Hu, R Ak, Y Xia, W Xu, A-H Sato, S Rachuri, L Ungar, PS Yu, R Govindaraju & T Suzumura (eds), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7841078, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016, Institute of Electrical and Electronics Engineers Inc., pp. 3968-3970, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 12/5/16. https://doi.org/10.1109/BigData.2016.7841078

Next-gen tools for big scientific data: ARM data center example. / Devarakonda, Ranjeet; Dumas, Kyle; Beus, Sheman et al.
Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. ed. / James Joshi; George Karypis; Ling Liu; Xiaohua Tony Hu; Ronay Ak; Yinglong Xia; Weijia Xu; Aki-Hiro Sato; Sudarsan Rachuri; Lyle Ungar; Philip S. Yu; Rama Govindaraju; Toyotaro Suzumura. Institute of Electrical and Electronics Engineers Inc., 2016. p. 3968-3970 7841078 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Next-gen tools for big scientific data

T2 - 4th IEEE International Conference on Big Data, Big Data 2016

AU - Devarakonda, Ranjeet

AU - Dumas, Kyle

AU - Beus, Sheman

AU - Rush, Everett

AU - Krishna, Bhargavi

AU - Records, Rob

AU - Prakash, Giri

PY - 2016

Y1 - 2016

N2 - The Atmospheric Radiation Measurement (ARM) Climate Research Facility (www.arm.gov) provides atmospheric observations from diverse climatic regimes around the world. Currently, ARM archives over 22 million user assessable data files, primarily stored in NetCDF file format, with total data volumes close to one Petabyte. In this paper, we will discuss how ARM is currently storing, distributing, cataloging and visualizing such large volumes of multi-dimensional climate observations and model data and also describe their future plan.

AB - The Atmospheric Radiation Measurement (ARM) Climate Research Facility (www.arm.gov) provides atmospheric observations from diverse climatic regimes around the world. Currently, ARM archives over 22 million user assessable data files, primarily stored in NetCDF file format, with total data volumes close to one Petabyte. In this paper, we will discuss how ARM is currently storing, distributing, cataloging and visualizing such large volumes of multi-dimensional climate observations and model data and also describe their future plan.

KW - ARM Archive

KW - Big Data Discovery

KW - Metadata Management

UR - https://www.scopus.com/pages/publications/85015233633

U2 - 10.1109/BigData.2016.7841078

DO - 10.1109/BigData.2016.7841078

M3 - Conference contribution

AN - SCOPUS:85015233633

T3 - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

SP - 3968

EP - 3970

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

A2 - Joshi, James

A2 - Karypis, George

A2 - Liu, Ling

A2 - Hu, Xiaohua Tony

A2 - Ak, Ronay

A2 - Xia, Yinglong

A2 - Xu, Weijia

A2 - Sato, Aki-Hiro

A2 - Rachuri, Sudarsan

A2 - Ungar, Lyle

A2 - Yu, Philip S.

A2 - Govindaraju, Rama

A2 - Suzumura, Toyotaro

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 5 December 2016 through 8 December 2016

ER -

Devarakonda R, Dumas K, Beus S, Rush E, Krishna B , Records R et al. Next-gen tools for big scientific data: ARM data center example. In Joshi J, Karypis G, Liu L, Hu XT, Ak R, Xia Y, Xu W, Sato AH, Rachuri S, Ungar L, Yu PS, Govindaraju R, Suzumura T, editors, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 3968-3970. 7841078. (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). doi: 10.1109/BigData.2016.7841078

Next-gen tools for big scientific data: ARM data center example

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this