Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

S. Fiore; M. Plociennik; C. Doutriaux; C. Palazzo; J. Boutte; T. Zok; D. Elia; M. Owsiak; A. D'Anca; Z. Shaheen; R. Bruno; M. Fargetta; M. Caballer; G. Molto; I. Blanquer; R. Barbera; M. David; G. Donvito; D. N. Williams; V. Anantharaj; D. Salomoni; G. Aloisio

doi:10.1109/BigData.2016.7840941

Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

S. Fiore, M. Plociennik, C. Doutriaux, C. Palazzo, J. Boutte, T. Zok, D. Elia, M. Owsiak, A. D'Anca, Z. Shaheen, R. Bruno, M. Fargetta, M. Caballer, G. Molto, I. Blanquer, R. Barbera, M. David, G. Donvito, D. N. Williams, V. AnantharajD. Salomoni, G. Aloisio

Data Lifecycle Technologies

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).

Original language	English
Title of host publication	Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
Editors	Ronay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2911-2918
Number of pages	8
ISBN (Electronic)	9781467390040
DOIs	https://doi.org/10.1109/BigData.2016.7840941
State	Published - 2016
Event	4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States Duration: Dec 5 2016 → Dec 8 2016

Publication series

Name	Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

Conference

Conference	4th IEEE International Conference on Big Data, Big Data 2016
Country/Territory	United States
City	Washington
Period	12/5/16 → 12/8/16

Funding

Funders	Funder number
Horizon 2020 Framework Programme	690116

Keywords

ESGF
INDIGO-DataCloud
big analytics
cloud computing
workflow management

Access to Document

10.1109/BigData.2016.7840941

Cite this

Fiore, S., Plociennik, M., Doutriaux, C., Palazzo, C., Boutte, J., Zok, T., Elia, D., Owsiak, M., D'Anca, A., Shaheen, Z., Bruno, R., Fargetta, M., Caballer, M., Molto, G., Blanquer, I., Barbera, R., David, M., Donvito, G., Williams, D. N., ... Aloisio, G. (2016). Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system. In R. Ak, G. Karypis, Y. Xia, X. T. Hu, P. S. Yu, J. Joshi, L. Ungar, L. Liu, A.-H. Sato, T. Suzumura, S. Rachuri, R. Govindaraju, & W. Xu (Eds.), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 2911-2918). Article 7840941 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7840941

Fiore, S. ; Plociennik, M. ; Doutriaux, C. et al. / Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. editor / Ronay Ak ; George Karypis ; Yinglong Xia ; Xiaohua Tony Hu ; Philip S. Yu ; James Joshi ; Lyle Ungar ; Ling Liu ; Aki-Hiro Sato ; Toyotaro Suzumura ; Sudarsan Rachuri ; Rama Govindaraju ; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 2911-2918 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

@inproceedings{cf005055b99340c2aef516c39b2eee70,

title = "Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system",

abstract = "A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).",

keywords = "ESGF, INDIGO-DataCloud, big analytics, cloud computing, workflow management",

author = "S. Fiore and M. Plociennik and C. Doutriaux and C. Palazzo and J. Boutte and T. Zok and D. Elia and M. Owsiak and A. D'Anca and Z. Shaheen and R. Bruno and M. Fargetta and M. Caballer and G. Molto and I. Blanquer and R. Barbera and M. David and G. Donvito and Williams, {D. N.} and V. Anantharaj and D. Salomoni and G. Aloisio",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 4th IEEE International Conference on Big Data, Big Data 2016 ; Conference date: 05-12-2016 Through 08-12-2016",

year = "2016",

doi = "10.1109/BigData.2016.7840941",

language = "English",

series = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2911--2918",

editor = "Ronay Ak and George Karypis and Yinglong Xia and Hu, {Xiaohua Tony} and Yu, {Philip S.} and James Joshi and Lyle Ungar and Ling Liu and Aki-Hiro Sato and Toyotaro Suzumura and Sudarsan Rachuri and Rama Govindaraju and Weijia Xu",

booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

}

Fiore, S, Plociennik, M, Doutriaux, C, Palazzo, C, Boutte, J, Zok, T, Elia, D, Owsiak, M, D'Anca, A, Shaheen, Z, Bruno, R, Fargetta, M, Caballer, M, Molto, G, Blanquer, I, Barbera, R, David, M, Donvito, G, Williams, DN, Anantharaj, V, Salomoni, D & Aloisio, G 2016, Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system. in R Ak, G Karypis, Y Xia, XT Hu, PS Yu, J Joshi, L Ungar, L Liu, A-H Sato, T Suzumura, S Rachuri, R Govindaraju & W Xu (eds), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7840941, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016, Institute of Electrical and Electronics Engineers Inc., pp. 2911-2918, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 12/5/16. https://doi.org/10.1109/BigData.2016.7840941

Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system. / Fiore, S.; Plociennik, M.; Doutriaux, C. et al.
Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. ed. / Ronay Ak; George Karypis; Yinglong Xia; Xiaohua Tony Hu; Philip S. Yu; James Joshi; Lyle Ungar; Ling Liu; Aki-Hiro Sato; Toyotaro Suzumura; Sudarsan Rachuri; Rama Govindaraju; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. p. 2911-2918 7840941 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

AU - Fiore, S.

AU - Plociennik, M.

AU - Doutriaux, C.

AU - Palazzo, C.

AU - Boutte, J.

AU - Zok, T.

AU - Elia, D.

AU - Owsiak, M.

AU - D'Anca, A.

AU - Shaheen, Z.

AU - Bruno, R.

AU - Fargetta, M.

AU - Caballer, M.

AU - Molto, G.

AU - Blanquer, I.

AU - Barbera, R.

AU - David, M.

AU - Donvito, G.

AU - Williams, D. N.

AU - Anantharaj, V.

AU - Salomoni, D.

AU - Aloisio, G.

PY - 2016

Y1 - 2016

N2 - A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).

AB - A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).

KW - ESGF

KW - INDIGO-DataCloud

KW - big analytics

KW - cloud computing

KW - workflow management

UR - http://www.scopus.com/inward/record.url?scp=85015232468&partnerID=8YFLogxK

U2 - 10.1109/BigData.2016.7840941

DO - 10.1109/BigData.2016.7840941

M3 - Conference contribution

AN - SCOPUS:85015232468

T3 - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

SP - 2911

EP - 2918

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

A2 - Ak, Ronay

A2 - Karypis, George

A2 - Xia, Yinglong

A2 - Hu, Xiaohua Tony

A2 - Yu, Philip S.

A2 - Joshi, James

A2 - Ungar, Lyle

A2 - Liu, Ling

A2 - Sato, Aki-Hiro

A2 - Suzumura, Toyotaro

A2 - Rachuri, Sudarsan

A2 - Govindaraju, Rama

A2 - Xu, Weijia

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th IEEE International Conference on Big Data, Big Data 2016

Y2 - 5 December 2016 through 8 December 2016

ER -

Fiore S, Plociennik M, Doutriaux C, Palazzo C, Boutte J, Zok T et al. Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system. In Ak R, Karypis G, Xia Y, Hu XT, Yu PS, Joshi J, Ungar L, Liu L, Sato AH, Suzumura T, Rachuri S, Govindaraju R, Xu W, editors, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 2911-2918. 7840941. (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). doi: 10.1109/BigData.2016.7840941

Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this