DES science portal: Creating science-ready catalogs

A. Fausti Neto; L. N. da Costa; A. Carnero; J. Gschwend; R. L.C. Ogando; F. Sobreira; M. A.G. Maia; B. X. Santiago; R. Rosenfeld; C. Singulani; C. Adean; L. D.P. Nunes; R. Campisano; R. Brito; G. Soares; G. C. Vila-Verde; T. M.C. Abbott; F. B. Abdalla; S. Allam; A. Benoit-Lévy; D. Brooks; E. Buckley-Geer; D. Capozzi; M. Carrasco Kind; J. Carretero; C. B. D'Andrea; S. Desai; P. Doel; A. Drlica-Wagner; A. E. Evrard; P. Fosalba; J. García-Bellido; D. W. Gerdes; R. A. Gruendl; G. Gutierrez; K. Honscheid; D. J. James; T. E. Jeltema; K. Kuehn; S. Kuhlmann; N. Kuropatkin; O. Lahav; M. Lima; J. L. Marshall; P. Melchior; F. Menanteau; A. Plazas; E. Sanchez; V. Scarpine; R. Schindler; M. Schubnell; I. Sevilla-Noarbe; M. Smith; R. C. Smith; E. Suchyta; M. E.C. Swanson; G. Tarle; A. R. Walker

doi:10.1016/j.ascom.2018.01.002

DES science portal: Creating science-ready catalogs

A. Fausti Neto, L. N. da Costa, A. Carnero, J. Gschwend, R. L.C. Ogando, F. Sobreira, M. A.G. Maia, B. X. Santiago, R. Rosenfeld, C. Singulani, C. Adean, L. D.P. Nunes, R. Campisano, R. Brito, G. Soares, G. C. Vila-Verde, T. M.C. Abbott, F. B. Abdalla, S. Allam, A. Benoit-LévyD. Brooks, E. Buckley-Geer, D. Capozzi, M. Carrasco Kind, J. Carretero, C. B. D'Andrea, S. Desai, P. Doel, A. Drlica-Wagner, A. E. Evrard, P. Fosalba, J. García-Bellido, D. W. Gerdes, R. A. Gruendl, G. Gutierrez, K. Honscheid, D. J. James, T. E. Jeltema, K. Kuehn, S. Kuhlmann, N. Kuropatkin, O. Lahav, M. Lima, J. L. Marshall, P. Melchior, F. Menanteau, A. Plazas, E. Sanchez, V. Scarpine, R. Schindler, M. Schubnell, I. Sevilla-Noarbe, M. Smith, R. C. Smith, E. Suchyta, M. E.C. Swanson, G. Tarle, A. R. Walker

Workflow Systems

Research output: Contribution to journal › Article › peer-review

5 Scopus citations

Abstract

We present a novel approach for creating science-ready catalogs through a software infrastructure developed for the Dark Energy Survey (DES). We integrate the data products released by the DES Data Management and additional products created by the DES collaboration in an environment known as DES Science Portal. Each step involved in the creation of a science-ready catalog is recorded in a relational database and can be recovered at any time. We describe how the DES Science Portal automates the creation and characterization of lightweight catalogs for DES Year 1 Annual Release, and show its flexibility in creating multiple catalogs with different inputs and configurations. Finally, we discuss the advantages of this infrastructure for large surveys such as DES and the Large Synoptic Survey Telescope. The capability of creating science-ready catalogs efficiently and with full control of the inputs and configurations used is an important asset for supporting science analysis using data from large astronomical surveys.

Original language	English
Pages (from-to)	52-69
Number of pages	18
Journal	Astronomy and Computing
Volume	24
DOIs	https://doi.org/10.1016/j.ascom.2018.01.002
State	Published - Jul 2018

Funding

Based in part on observations at Cerro Tololo Inter-American Observatory, National Optical Astronomy Observatory, which is operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation. Funding for the DES Projects has been provided by the U.S. Department of Energy, the U.S. National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the Higher Education Funding Council for England, the National Center for Supercomputing Applications at the University of Illinois at Urbana–Champaign, the Kavli Institute of Cosmological Physics at the University of Chicago, the Center for Cosmology and Astro-Particle Physics at the Ohio State University, the Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University, Financiadora de Estudos e Projetos, Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro, Conselho Nacional de Desenvolvimento Científico e Tecnológico and the Ministério da Ciência, Tecnologia e Inovação, the Deutsche Forschungsgemeinschaft and the Collaborating Institutions in the Dark Energy Survey. The DES data management system is supported by the National Science Foundation under Grant Numbers AST-1138766 and AST-1536171 . The DES participants from Spanish institutions are partially supported by MINECO under grants AYA2015-71825 , ESP2015-88861 , FPA2015-68048 , SEV-2012-0234 , SEV-2016-0597 , and MDM-2015-0509 , some of which include ERDF funds from the European Union. IFAE is partially funded by the CERCA program of the Generalitat de Catalunya . Research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Program (FP7/2007–2013) including ERC grant agreements 240672 , 291329 , and 306478 . We acknowledge support from the Australian Research Council Centre of Excellence for All-sky Astrophysics (CAASTRO) , through project number CE110001020 . This manuscript has been authored by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U.S. Department of Energy, Office of Science, Office of High Energy Physics. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. JG is supported by CAPES . ACR is supported by CNPq grant 157684/2015-6 .

Funders	Funder number
Collaborating Institutions in the Dark Energy Survey
Fermi Research Alliance, LLC	DE-AC02-07CH11359
Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University
Science and Technology Facilities Council of the United Kingdom
National Science Foundation	AST-1138766, AST-1536171
U.S. Department of Energy
Office of Science
High Energy Physics
Ohio State University
University of Chicago
Seventh Framework Programme	1138766, 240672, 306478, 291329
Higher Education Funding Council for England
European Commission
European Research Council
Australian Research Council	CE110001020
Deutsche Forschungsgemeinschaft
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Generalitat de Catalunya
Ministerio de Economía y Competitividad	ESP2015-88861, SEV-2016-0597, MDM-2015-0509, FPA2015-68048, SEV-2012-0234, AYA2015-71825
Ministério da Ciência, Tecnologia e Inovação
Conselho Nacional de Desenvolvimento Científico e Tecnológico	157684/2015-6
Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro
Financiadora de Estudos e Projetos
Ministry of Education and Science of Ukraine
European Regional Development Fund

Keywords

Astronomical databases
Catalogs
Data analysis
Surveys—methods

Access to Document

10.1016/j.ascom.2018.01.002

Cite this

Fausti Neto, A., da Costa, L. N., Carnero, A., Gschwend, J., Ogando, R. L. C., Sobreira, F., Maia, M. A. G., Santiago, B. X., Rosenfeld, R., Singulani, C., Adean, C., Nunes, L. D. P., Campisano, R., Brito, R., Soares, G., Vila-Verde, G. C., Abbott, T. M. C., Abdalla, F. B., Allam, S., ... Walker, A. R. (2018). DES science portal: Creating science-ready catalogs. Astronomy and Computing, 24, 52-69. https://doi.org/10.1016/j.ascom.2018.01.002

@article{5f54ccd2580a4627875320b95d708cb5,

title = "DES science portal: Creating science-ready catalogs",

abstract = "We present a novel approach for creating science-ready catalogs through a software infrastructure developed for the Dark Energy Survey (DES). We integrate the data products released by the DES Data Management and additional products created by the DES collaboration in an environment known as DES Science Portal. Each step involved in the creation of a science-ready catalog is recorded in a relational database and can be recovered at any time. We describe how the DES Science Portal automates the creation and characterization of lightweight catalogs for DES Year 1 Annual Release, and show its flexibility in creating multiple catalogs with different inputs and configurations. Finally, we discuss the advantages of this infrastructure for large surveys such as DES and the Large Synoptic Survey Telescope. The capability of creating science-ready catalogs efficiently and with full control of the inputs and configurations used is an important asset for supporting science analysis using data from large astronomical surveys.",

keywords = "Astronomical databases, Catalogs, Data analysis, Surveys—methods",

author = "{Fausti Neto}, A. and {da Costa}, {L. N.} and A. Carnero and J. Gschwend and Ogando, {R. L.C.} and F. Sobreira and Maia, {M. A.G.} and Santiago, {B. X.} and R. Rosenfeld and C. Singulani and C. Adean and Nunes, {L. D.P.} and R. Campisano and R. Brito and G. Soares and Vila-Verde, {G. C.} and Abbott, {T. M.C.} and Abdalla, {F. B.} and S. Allam and A. Benoit-L{\'e}vy and D. Brooks and E. Buckley-Geer and D. Capozzi and {Carrasco Kind}, M. and J. Carretero and D'Andrea, {C. B.} and S. Desai and P. Doel and A. Drlica-Wagner and Evrard, {A. E.} and P. Fosalba and J. Garc{\'i}a-Bellido and Gerdes, {D. W.} and Gruendl, {R. A.} and G. Gutierrez and K. Honscheid and James, {D. J.} and Jeltema, {T. E.} and K. Kuehn and S. Kuhlmann and N. Kuropatkin and O. Lahav and M. Lima and Marshall, {J. L.} and P. Melchior and F. Menanteau and A. Plazas and E. Sanchez and V. Scarpine and R. Schindler and M. Schubnell and I. Sevilla-Noarbe and M. Smith and Smith, {R. C.} and E. Suchyta and Swanson, {M. E.C.} and G. Tarle and Walker, {A. R.}",

note = "Publisher Copyright: {\textcopyright} 2018 Elsevier B.V.",

year = "2018",

month = jul,

doi = "10.1016/j.ascom.2018.01.002",

language = "English",

volume = "24",

pages = "52--69",

journal = "Astronomy and Computing",

issn = "2213-1337",

}

Fausti Neto, A, da Costa, LN, Carnero, A, Gschwend, J, Ogando, RLC, Sobreira, F, Maia, MAG, Santiago, BX, Rosenfeld, R, Singulani, C, Adean, C, Nunes, LDP, Campisano, R, Brito, R, Soares, G, Vila-Verde, GC, Abbott, TMC, Abdalla, FB, Allam, S, Benoit-Lévy, A, Brooks, D, Buckley-Geer, E, Capozzi, D, Carrasco Kind, M, Carretero, J, D'Andrea, CB, Desai, S, Doel, P, Drlica-Wagner, A, Evrard, AE, Fosalba, P, García-Bellido, J, Gerdes, DW, Gruendl, RA, Gutierrez, G, Honscheid, K, James, DJ, Jeltema, TE, Kuehn, K, Kuhlmann, S, Kuropatkin, N, Lahav, O, Lima, M, Marshall, JL, Melchior, P, Menanteau, F, Plazas, A, Sanchez, E, Scarpine, V, Schindler, R, Schubnell, M, Sevilla-Noarbe, I, Smith, M, Smith, RC, Suchyta, E, Swanson, MEC, Tarle, G & Walker, AR 2018, 'DES science portal: Creating science-ready catalogs', Astronomy and Computing, vol. 24, pp. 52-69. https://doi.org/10.1016/j.ascom.2018.01.002

TY - JOUR

T1 - DES science portal

T2 - Creating science-ready catalogs

AU - Fausti Neto, A.

AU - da Costa, L. N.

AU - Carnero, A.

AU - Gschwend, J.

AU - Ogando, R. L.C.

AU - Sobreira, F.

AU - Maia, M. A.G.

AU - Santiago, B. X.

AU - Rosenfeld, R.

AU - Singulani, C.

AU - Adean, C.

AU - Nunes, L. D.P.

AU - Campisano, R.

AU - Brito, R.

AU - Soares, G.

AU - Vila-Verde, G. C.

AU - Abbott, T. M.C.

AU - Abdalla, F. B.

AU - Allam, S.

AU - Benoit-Lévy, A.

AU - Brooks, D.

AU - Buckley-Geer, E.

AU - Capozzi, D.

AU - Carrasco Kind, M.

AU - Carretero, J.

AU - D'Andrea, C. B.

AU - Desai, S.

AU - Doel, P.

AU - Drlica-Wagner, A.

AU - Evrard, A. E.

AU - Fosalba, P.

AU - García-Bellido, J.

AU - Gerdes, D. W.

AU - Gruendl, R. A.

AU - Gutierrez, G.

AU - Honscheid, K.

AU - James, D. J.

AU - Jeltema, T. E.

AU - Kuehn, K.

AU - Kuhlmann, S.

AU - Kuropatkin, N.

AU - Lahav, O.

AU - Lima, M.

AU - Marshall, J. L.

AU - Melchior, P.

AU - Menanteau, F.

AU - Plazas, A.

AU - Sanchez, E.

AU - Scarpine, V.

AU - Schindler, R.

AU - Schubnell, M.

AU - Sevilla-Noarbe, I.

AU - Smith, M.

AU - Smith, R. C.

AU - Suchyta, E.

AU - Swanson, M. E.C.

AU - Tarle, G.

AU - Walker, A. R.

PY - 2018/7

Y1 - 2018/7

N2 - We present a novel approach for creating science-ready catalogs through a software infrastructure developed for the Dark Energy Survey (DES). We integrate the data products released by the DES Data Management and additional products created by the DES collaboration in an environment known as DES Science Portal. Each step involved in the creation of a science-ready catalog is recorded in a relational database and can be recovered at any time. We describe how the DES Science Portal automates the creation and characterization of lightweight catalogs for DES Year 1 Annual Release, and show its flexibility in creating multiple catalogs with different inputs and configurations. Finally, we discuss the advantages of this infrastructure for large surveys such as DES and the Large Synoptic Survey Telescope. The capability of creating science-ready catalogs efficiently and with full control of the inputs and configurations used is an important asset for supporting science analysis using data from large astronomical surveys.

AB - We present a novel approach for creating science-ready catalogs through a software infrastructure developed for the Dark Energy Survey (DES). We integrate the data products released by the DES Data Management and additional products created by the DES collaboration in an environment known as DES Science Portal. Each step involved in the creation of a science-ready catalog is recorded in a relational database and can be recovered at any time. We describe how the DES Science Portal automates the creation and characterization of lightweight catalogs for DES Year 1 Annual Release, and show its flexibility in creating multiple catalogs with different inputs and configurations. Finally, we discuss the advantages of this infrastructure for large surveys such as DES and the Large Synoptic Survey Telescope. The capability of creating science-ready catalogs efficiently and with full control of the inputs and configurations used is an important asset for supporting science analysis using data from large astronomical surveys.

KW - Astronomical databases

KW - Catalogs

KW - Data analysis

KW - Surveys—methods

UR - http://www.scopus.com/inward/record.url?scp=85049117308&partnerID=8YFLogxK

U2 - 10.1016/j.ascom.2018.01.002

DO - 10.1016/j.ascom.2018.01.002

M3 - Article

AN - SCOPUS:85049117308

SN - 2213-1337

VL - 24

SP - 52

EP - 69

JO - Astronomy and Computing

JF - Astronomy and Computing

ER -

DES science portal: Creating science-ready catalogs

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this