DES science portal: Computing photometric redshifts

J. Gschwend; A. C. Rossel; R. L.C. Ogando; A. F. Neto; M. A.G. Maia; L. N. da Costa; M. Lima; P. Pellegrini; R. Campisano; C. Singulani; C. Adean; C. Benoist; M. Aguena; M. Carrasco Kind; T. M. Davis; J. de Vicente; W. G. Hartley; B. Hoyle; A. Palmese; I. Sadeh; T. M.C. Abbott; F. B. Abdalla; S. Allam; J. Annis; J. Asorey; D. Brooks; J. Calcino; D. Carollo; F. J. Castander; C. B. D'Andrea; S. Desai; A. E. Evrard; P. Fosalba; J. Frieman; J. García-Bellido; K. Glazebrook; D. W. Gerdes; R. A. Gruendl; G. Gutierrez; S. Hinton; D. L. Hollowood; K. Honscheid; J. K. Hoormann; D. J. James; K. Kuehn; N. Kuropatkin; O. Lahav; G. Lewis; C. Lidman; H. Lin; E. Macaulay; J. Marshall; P. Melchior; R. Miquel; A. Möller; A. A. Plazas; E. Sanchez; B. Santiago; V. Scarpine; R. H. Schindler; I. Sevilla-Noarbe; M. Smith; F. Sobreira; N. E. Sommer; E. Suchyta; M. E.C. Swanson; G. Tarle; B. E. Tucker; D. L. Tucker; S. Uddin; A. R. Walker

doi:10.1016/j.ascom.2018.08.008

DES science portal: Computing photometric redshifts

J. Gschwend, A. C. Rossel, R. L.C. Ogando, A. F. Neto, M. A.G. Maia, L. N. da Costa, M. Lima, P. Pellegrini, R. Campisano, C. Singulani, C. Adean, C. Benoist, M. Aguena, M. Carrasco Kind, T. M. Davis, J. de Vicente, W. G. Hartley, B. Hoyle, A. Palmese, I. SadehT. M.C. Abbott, F. B. Abdalla, S. Allam, J. Annis, J. Asorey, D. Brooks, J. Calcino, D. Carollo, F. J. Castander, C. B. D'Andrea, S. Desai, A. E. Evrard, P. Fosalba, J. Frieman, J. García-Bellido, K. Glazebrook, D. W. Gerdes, R. A. Gruendl, G. Gutierrez, S. Hinton, D. L. Hollowood, K. Honscheid, J. K. Hoormann, D. J. James, K. Kuehn, N. Kuropatkin, O. Lahav, G. Lewis, C. Lidman, H. Lin, E. Macaulay, J. Marshall, P. Melchior, R. Miquel, A. Möller, A. A. Plazas, E. Sanchez, B. Santiago, V. Scarpine, R. H. Schindler, I. Sevilla-Noarbe, M. Smith, F. Sobreira, N. E. Sommer, E. Suchyta, M. E.C. Swanson, G. Tarle, B. E. Tucker, D. L. Tucker, S. Uddin, A. R. Walker

Workflow Systems

Research output: Contribution to journal › Article › peer-review

21 Scopus citations

Abstract

A significant challenge facing photometric surveys for cosmological purposes is the need to produce reliable redshift estimates. The estimation of photometric redshifts (photo-zs) has been consolidated as the standard strategy to bypass the high production costs and incompleteness of spectroscopic redshift samples. Training-based photo-z methods require the preparation of a high-quality list of spectroscopic redshifts, which needs to be constantly updated. The photo-z training, validation, and estimation must be performed in a consistent and reproducible way in order to accomplish the scientific requirements. To meet this purpose, we developed an integrated web-based data interface that not only provides the framework to carry out the above steps in a systematic way, enabling the ease testing and comparison of different algorithms, but also addresses the processing requirements by parallelizing the calculation in a transparent way for the user. This framework called the Science Portal (hereafter Portal) was developed in the context the Dark Energy Survey (DES) to facilitate scientific analysis. In this paper, we show how the Portal can provide a reliable environment to access vast datasets, provide validation algorithms and metrics, even in the case of multiple photo-zs methods. It is possible to maintain the provenance between the steps of a chain of workflows while ensuring reproducibility of the results. We illustrate how the Portal can be used to provide photo-z estimates using the DES first year (Y1A1) data. While the DES collaboration is still developing techniques to obtain more precise photo-zs, having a structured framework like the one presented here is critical for the systematic vetting of DES algorithmic improvements and the consistent production of photo-zs in future DES releases.

Original language	English
Pages (from-to)	58-80
Number of pages	23
Journal	Astronomy and Computing
Volume	25
DOIs	https://doi.org/10.1016/j.ascom.2018.08.008
State	Published - Oct 2018

Funding

JG is supported by CAPES , Brazil process 1454767 . ACR is supported by CNPq , Brazil process 157684/2015-6 . ML is partially supported by CNPq , Brazil and FAPESP , Brazil. MA is supported by CNPq , Brazil process 165049/2017-0 . Part of this research is supported by INCT do e–Universo , Brazil (CNPq grants 465376/2014-2 ). Based in part on observations at Cerro Tololo Inter-American Observatory, National Optical Astronomy Observatory, which is operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation. The DES data management system is supported by the National Science Foundation, United States under Grant Numbers AST-1138766 and AST-1536171. The DES participants from Spanish institutions are partially supported by MINECO, Spain under grants AYA201571825, ESP2015-66861, FPA2015-68048, SEV-2016-0588, SEV-2016-0597, and MDM-2015-0509, some of which include ERDF funds from the European Union. IFAE is partially funded by the CERCA program of the Generalitat de Catalunya, Spain. Research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Program (FP7/2007-2013), Spain including ERC grant agreements 240672, 291329, and 306478. We acknowledge support from the Australian Research Council Centre of Excellence for All-sky Astrophysics (CAASTRO), through project number CE110001020, and the Brazilian Instituto Nacional de Ciência e Tecnologia (INCT) e-Universe (CNPq grant 465376/2014-2). Funding for the DES Projects has been provided by the U.S. Department of Energy , the U.S. National Science Foundation , the Ministry of Science and Education of Spain , the Science and Technology Facilities Council of the United Kingdom , the Higher Education Funding Council for England , United Kingdom, the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign , United States, the Kavli Institute of Cosmological Physics at the University of Chicago , United States, the Center for Cosmology and Astro-Particle Physics at the Ohio State University , United States, the Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University , United States, Financiadora de Estudos e Projetos , Brazil, Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro , Brazil, Conselho Nacional de Desenvolvimento Científico e Tecnológico , Brazil and the Ministério da Ciência, Tecnologia e Inovação , Brazil, the Deutsche Forschungsgemeinschaft , Germany and the Collaborating Institutions in the Dark Energy Survey. The DES data management system is supported by the National Science Foundation , United States under Grant Numbers AST-1138766 and AST-1536171 . The DES participants from Spanish institutions are partially supported by MINECO , Spain under grants AYA201571825 , ESP2015-66861 , FPA2015-68048 , SEV-2016-0588 , SEV-2016-0597 , and MDM-2015-0509 , some of which include ERDF funds from the European Union. IFAE is partially funded by the CERCA program of the Generalitat de Catalunya , Spain. Research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013) , Spain including ERC grant agreements 240672 , 291329 , and 306478 . We acknowledge support from the Australian Research Council Centre of Excellence for All-sky Astrophysics (CAASTRO) , through project number CE110001020, and the Brazilian Instituto Nacional de Ciência e Tecnologia (INCT) e-Universe (CNPq grant 465376/2014-2 ). This manuscript has been authored by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U.S. Department of Energy, Office of Science, Office of High Energy Physics. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.

Funders	Funder number
Australian Research Council Centre of Excellence for All-sky Astrophysics	CE110001020
Collaborating Institutions in the Dark Energy Survey
European Union’s Seventh Framework Program	FP7/2007-2013
Fermi Research Alliance, LLC	DE-AC02-07CH11359
INCT
Kavli Institute of Cosmological Physics at the University of Chicago
Ministry of Science and Education of Spain
Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University
U.S. National Science Foundation
National Science Foundation	SEV-2016-0588, SEV-2016-0597, ESP2015-66861, AST-1138766, MDM-2015-0509, FPA2015-68048, AST-1536171, AYA201571825
U.S. Department of Energy
Directorate for Mathematical and Physical Sciences	1138766
University of Illinois at Urbana-Champaign
Texas Sea Grant, Texas A and M University
Office of Science
High Energy Physics
Ohio State University
Higher Education Funding Council for England
Center for Cosmology and Astroparticle Physics, Ohio State University
Instituto Nacional de Ciência e Tecnologia Midas
Science and Technology Facilities Council
Higher Education Funding Council for England
European Commission
European Research Council	240672, 306478, 291329
Deutsche Forschungsgemeinschaft
Fundação de Amparo à Pesquisa do Estado de São Paulo
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior	1454767
Generalitat de Catalunya
Ministerio de Economía y Competitividad
Ministério da Ciência, Tecnologia e Inovação
Conselho Nacional de Desenvolvimento Científico e Tecnológico	465376/2014-2, 157684/2015-6
Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro
Financiadora de Estudos e Projetos
European Regional Development Fund

Keywords

Astronomical databases: catalogs, surveys
Galaxies: distances and redshifts, statistics
Methods: data analysis

Access to Document

10.1016/j.ascom.2018.08.008

Cite this

Gschwend, J., Rossel, A. C., Ogando, R. L. C., Neto, A. F., Maia, M. A. G., da Costa, L. N., Lima, M., Pellegrini, P., Campisano, R., Singulani, C., Adean, C., Benoist, C., Aguena, M., Carrasco Kind, M., Davis, T. M., de Vicente, J., Hartley, W. G., Hoyle, B., Palmese, A., ... Walker, A. R. (2018). DES science portal: Computing photometric redshifts. Astronomy and Computing, 25, 58-80. https://doi.org/10.1016/j.ascom.2018.08.008

@article{8dea2ce0fafe43aa9baa87f5a83369a8,

title = "DES science portal: Computing photometric redshifts",

abstract = "A significant challenge facing photometric surveys for cosmological purposes is the need to produce reliable redshift estimates. The estimation of photometric redshifts (photo-zs) has been consolidated as the standard strategy to bypass the high production costs and incompleteness of spectroscopic redshift samples. Training-based photo-z methods require the preparation of a high-quality list of spectroscopic redshifts, which needs to be constantly updated. The photo-z training, validation, and estimation must be performed in a consistent and reproducible way in order to accomplish the scientific requirements. To meet this purpose, we developed an integrated web-based data interface that not only provides the framework to carry out the above steps in a systematic way, enabling the ease testing and comparison of different algorithms, but also addresses the processing requirements by parallelizing the calculation in a transparent way for the user. This framework called the Science Portal (hereafter Portal) was developed in the context the Dark Energy Survey (DES) to facilitate scientific analysis. In this paper, we show how the Portal can provide a reliable environment to access vast datasets, provide validation algorithms and metrics, even in the case of multiple photo-zs methods. It is possible to maintain the provenance between the steps of a chain of workflows while ensuring reproducibility of the results. We illustrate how the Portal can be used to provide photo-z estimates using the DES first year (Y1A1) data. While the DES collaboration is still developing techniques to obtain more precise photo-zs, having a structured framework like the one presented here is critical for the systematic vetting of DES algorithmic improvements and the consistent production of photo-zs in future DES releases.",

keywords = "Astronomical databases: catalogs, surveys, Galaxies: distances and redshifts, statistics, Methods: data analysis",

author = "J. Gschwend and Rossel, {A. C.} and Ogando, {R. L.C.} and Neto, {A. F.} and Maia, {M. A.G.} and {da Costa}, {L. N.} and M. Lima and P. Pellegrini and R. Campisano and C. Singulani and C. Adean and C. Benoist and M. Aguena and {Carrasco Kind}, M. and Davis, {T. M.} and {de Vicente}, J. and Hartley, {W. G.} and B. Hoyle and A. Palmese and I. Sadeh and Abbott, {T. M.C.} and Abdalla, {F. B.} and S. Allam and J. Annis and J. Asorey and D. Brooks and J. Calcino and D. Carollo and Castander, {F. J.} and D'Andrea, {C. B.} and S. Desai and Evrard, {A. E.} and P. Fosalba and J. Frieman and J. Garc{\'i}a-Bellido and K. Glazebrook and Gerdes, {D. W.} and Gruendl, {R. A.} and G. Gutierrez and S. Hinton and Hollowood, {D. L.} and K. Honscheid and Hoormann, {J. K.} and James, {D. J.} and K. Kuehn and N. Kuropatkin and O. Lahav and G. Lewis and C. Lidman and H. Lin and E. Macaulay and J. Marshall and P. Melchior and R. Miquel and A. M{\"o}ller and Plazas, {A. A.} and E. Sanchez and B. Santiago and V. Scarpine and Schindler, {R. H.} and I. Sevilla-Noarbe and M. Smith and F. Sobreira and Sommer, {N. E.} and E. Suchyta and Swanson, {M. E.C.} and G. Tarle and Tucker, {B. E.} and Tucker, {D. L.} and S. Uddin and Walker, {A. R.}",

note = "Publisher Copyright: {\textcopyright} 2018 Elsevier B.V.",

year = "2018",

month = oct,

doi = "10.1016/j.ascom.2018.08.008",

language = "English",

volume = "25",

pages = "58--80",

journal = "Astronomy and Computing",

issn = "2213-1337",

}

Gschwend, J, Rossel, AC, Ogando, RLC, Neto, AF, Maia, MAG, da Costa, LN, Lima, M, Pellegrini, P, Campisano, R, Singulani, C, Adean, C, Benoist, C, Aguena, M, Carrasco Kind, M, Davis, TM, de Vicente, J, Hartley, WG, Hoyle, B, Palmese, A, Sadeh, I, Abbott, TMC, Abdalla, FB, Allam, S, Annis, J, Asorey, J, Brooks, D, Calcino, J, Carollo, D, Castander, FJ, D'Andrea, CB, Desai, S, Evrard, AE, Fosalba, P, Frieman, J, García-Bellido, J, Glazebrook, K, Gerdes, DW, Gruendl, RA, Gutierrez, G, Hinton, S, Hollowood, DL, Honscheid, K, Hoormann, JK, James, DJ, Kuehn, K, Kuropatkin, N, Lahav, O, Lewis, G, Lidman, C, Lin, H, Macaulay, E, Marshall, J, Melchior, P, Miquel, R, Möller, A, Plazas, AA, Sanchez, E, Santiago, B, Scarpine, V, Schindler, RH, Sevilla-Noarbe, I, Smith, M, Sobreira, F, Sommer, NE, Suchyta, E, Swanson, MEC, Tarle, G, Tucker, BE, Tucker, DL, Uddin, S & Walker, AR 2018, 'DES science portal: Computing photometric redshifts', Astronomy and Computing, vol. 25, pp. 58-80. https://doi.org/10.1016/j.ascom.2018.08.008

TY - JOUR

T1 - DES science portal

T2 - Computing photometric redshifts

AU - Gschwend, J.

AU - Rossel, A. C.

AU - Ogando, R. L.C.

AU - Neto, A. F.

AU - Maia, M. A.G.

AU - da Costa, L. N.

AU - Lima, M.

AU - Pellegrini, P.

AU - Campisano, R.

AU - Singulani, C.

AU - Adean, C.

AU - Benoist, C.

AU - Aguena, M.

AU - Carrasco Kind, M.

AU - Davis, T. M.

AU - de Vicente, J.

AU - Hartley, W. G.

AU - Hoyle, B.

AU - Palmese, A.

AU - Sadeh, I.

AU - Abbott, T. M.C.

AU - Abdalla, F. B.

AU - Allam, S.

AU - Annis, J.

AU - Asorey, J.

AU - Brooks, D.

AU - Calcino, J.

AU - Carollo, D.

AU - Castander, F. J.

AU - D'Andrea, C. B.

AU - Desai, S.

AU - Evrard, A. E.

AU - Fosalba, P.

AU - Frieman, J.

AU - García-Bellido, J.

AU - Glazebrook, K.

AU - Gerdes, D. W.

AU - Gruendl, R. A.

AU - Gutierrez, G.

AU - Hinton, S.

AU - Hollowood, D. L.

AU - Honscheid, K.

AU - Hoormann, J. K.

AU - James, D. J.

AU - Kuehn, K.

AU - Kuropatkin, N.

AU - Lahav, O.

AU - Lewis, G.

AU - Lidman, C.

AU - Lin, H.

AU - Macaulay, E.

AU - Marshall, J.

AU - Melchior, P.

AU - Miquel, R.

AU - Möller, A.

AU - Plazas, A. A.

AU - Sanchez, E.

AU - Santiago, B.

AU - Scarpine, V.

AU - Schindler, R. H.

AU - Sevilla-Noarbe, I.

AU - Smith, M.

AU - Sobreira, F.

AU - Sommer, N. E.

AU - Suchyta, E.

AU - Swanson, M. E.C.

AU - Tarle, G.

AU - Tucker, B. E.

AU - Tucker, D. L.

AU - Uddin, S.

AU - Walker, A. R.

PY - 2018/10

Y1 - 2018/10

N2 - A significant challenge facing photometric surveys for cosmological purposes is the need to produce reliable redshift estimates. The estimation of photometric redshifts (photo-zs) has been consolidated as the standard strategy to bypass the high production costs and incompleteness of spectroscopic redshift samples. Training-based photo-z methods require the preparation of a high-quality list of spectroscopic redshifts, which needs to be constantly updated. The photo-z training, validation, and estimation must be performed in a consistent and reproducible way in order to accomplish the scientific requirements. To meet this purpose, we developed an integrated web-based data interface that not only provides the framework to carry out the above steps in a systematic way, enabling the ease testing and comparison of different algorithms, but also addresses the processing requirements by parallelizing the calculation in a transparent way for the user. This framework called the Science Portal (hereafter Portal) was developed in the context the Dark Energy Survey (DES) to facilitate scientific analysis. In this paper, we show how the Portal can provide a reliable environment to access vast datasets, provide validation algorithms and metrics, even in the case of multiple photo-zs methods. It is possible to maintain the provenance between the steps of a chain of workflows while ensuring reproducibility of the results. We illustrate how the Portal can be used to provide photo-z estimates using the DES first year (Y1A1) data. While the DES collaboration is still developing techniques to obtain more precise photo-zs, having a structured framework like the one presented here is critical for the systematic vetting of DES algorithmic improvements and the consistent production of photo-zs in future DES releases.

AB - A significant challenge facing photometric surveys for cosmological purposes is the need to produce reliable redshift estimates. The estimation of photometric redshifts (photo-zs) has been consolidated as the standard strategy to bypass the high production costs and incompleteness of spectroscopic redshift samples. Training-based photo-z methods require the preparation of a high-quality list of spectroscopic redshifts, which needs to be constantly updated. The photo-z training, validation, and estimation must be performed in a consistent and reproducible way in order to accomplish the scientific requirements. To meet this purpose, we developed an integrated web-based data interface that not only provides the framework to carry out the above steps in a systematic way, enabling the ease testing and comparison of different algorithms, but also addresses the processing requirements by parallelizing the calculation in a transparent way for the user. This framework called the Science Portal (hereafter Portal) was developed in the context the Dark Energy Survey (DES) to facilitate scientific analysis. In this paper, we show how the Portal can provide a reliable environment to access vast datasets, provide validation algorithms and metrics, even in the case of multiple photo-zs methods. It is possible to maintain the provenance between the steps of a chain of workflows while ensuring reproducibility of the results. We illustrate how the Portal can be used to provide photo-z estimates using the DES first year (Y1A1) data. While the DES collaboration is still developing techniques to obtain more precise photo-zs, having a structured framework like the one presented here is critical for the systematic vetting of DES algorithmic improvements and the consistent production of photo-zs in future DES releases.

KW - Astronomical databases: catalogs, surveys

KW - Galaxies: distances and redshifts, statistics

KW - Methods: data analysis

UR - http://www.scopus.com/inward/record.url?scp=85053382995&partnerID=8YFLogxK

U2 - 10.1016/j.ascom.2018.08.008

DO - 10.1016/j.ascom.2018.08.008

M3 - Article

AN - SCOPUS:85053382995

SN - 2213-1337

VL - 25

SP - 58

EP - 80

JO - Astronomy and Computing

JF - Astronomy and Computing

ER -

DES science portal: Computing photometric redshifts

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this