Blinding multiprobe cosmological experiments

J. Muir; G. M. Bernstein; D. Huterer; F. Elsner; E. Krause; A. Roodman; S. Allam; J. Annis; S. Avila; K. Bechtol; E. Bertin; D. Brooks; E. Buckley-Geer; D. L. Burke; A. Carnero Rosell; M. Carrasco Kind; J. Carretero; R. Cawthon; M. Costanzi; L. N. Da Costa; J. De Vicente; S. Desai; J. P. Dietrich; P. Doel; T. F. Eifler; S. Everett; P. Fosalba; J. Frieman; J. García-Bellido; D. W. Gerdes; D. Gruen; R. A. Gruendl; J. Gschwend; W. G. Hartley; D. L. Hollowood; D. J. James; M. Jarvis; K. Kuehn; N. Kuropatkin; O. Lahav; M. March; J. L. Marshall; P. Melchior; F. Menanteau; R. Miquel; R. L.C. Ogando; A. Palmese; F. Paz-Chinchón; A. A. Plazas; A. K. Romer; E. Sanchez; V. Scarpine; M. Schubnell; S. Serrano; I. Sevilla-Noarbe; M. Smith; E. Suchyta; G. Tarle; D. Thomas; M. A. Troxel; A. R. Walker; J. Weller; W. Wester; J. Zuntz

doi:10.1093/MNRAS/STAA965

Blinding multiprobe cosmological experiments

J. Muir, G. M. Bernstein, D. Huterer, F. Elsner, E. Krause, A. Roodman, S. Allam, J. Annis, S. Avila, K. Bechtol, E. Bertin, D. Brooks, E. Buckley-Geer, D. L. Burke, A. Carnero Rosell, M. Carrasco Kind, J. Carretero, R. Cawthon, M. Costanzi, L. N. Da CostaJ. De Vicente, S. Desai, J. P. Dietrich, P. Doel, T. F. Eifler, S. Everett, P. Fosalba, J. Frieman, J. García-Bellido, D. W. Gerdes, D. Gruen, R. A. Gruendl, J. Gschwend, W. G. Hartley, D. L. Hollowood, D. J. James, M. Jarvis, K. Kuehn, N. Kuropatkin, O. Lahav, M. March, J. L. Marshall, P. Melchior, F. Menanteau, R. Miquel, R. L.C. Ogando, A. Palmese, F. Paz-Chinchón, A. A. Plazas, A. K. Romer, E. Sanchez, V. Scarpine, M. Schubnell, S. Serrano, I. Sevilla-Noarbe, M. Smith, E. Suchyta, G. Tarle, D. Thomas, M. A. Troxel, A. R. Walker, J. Weller, W. Wester, J. Zuntz

Workflow Systems

Research output: Contribution to journal › Article › peer-review

25 Scopus citations

Abstract

The goal of blinding is to hide an experiment's critical results - here the inferred cosmological parameters - until all decisions affecting its analysis have been finalized. This is especially important in the current era of precision cosmology,when the results of any newexperiment are closely scrutinized for consistency or tension with previous results. In analyses that combine multiple observational probes, like the combination of galaxy clustering and weak lensing in the Dark Energy Survey (DES), it is challenging to blind the results while retaining the ability to check for (in)consistency between different parts of the data. We propose a simple new blinding transformation, which works by modifying the summary statistics that are input to parameter estimation, such as two-point correlation functions. The transformation shifts the measured statistics to new values that are consistent with (blindly) shifted cosmological parameters while preserving internal (in)consistency. We apply the blinding transformation to simulated data for the projected DES Year 3 galaxy clustering and weak lensing analysis, demonstrating that practical blinding is achieved without significant perturbation of internalconsistency checks, asmeasured here by degradation of the χ2 between the data and best-fitting model. Our blinding method's performance is expected to improve as experiments evolve to higher precision and accuracy.

Original language	English
Pages (from-to)	4454-4470
Number of pages	17
Journal	Monthly Notices of the Royal Astronomical Society
Volume	494
Issue number	3
DOIs	https://doi.org/10.1093/MNRAS/STAA965
State	Published - 2020

Funding

The analysis made use of the software tools scipy (Jones et al. 2001), numpy (Oliphant 2006), matplotlib (Hunter 2007), getdist (Lewis 2019), MULTINEST (Feroz & Hobson 2008; Feroz et al. 2009, 2013), cosmosis (Zuntz et al. 2015), and cosmolike (Krause & Eifler 2017). It was supported in part through computational resources and services provided by Advanced Research Computing at the National Science Foundation, Ann Arbor; the National Energy Research Scientific Computing Center (NERSC), a US Department of Energy Office of Science User Facility operated under Contract No. DE-AC02-05CH11231; and the Sherlock cluster, supported by Stanford University and the Stanford Research Computing Center. We would like to thank all of these facilities for providing computational resources and support that contributed to these research results. JM has been supported by the Porat Fellowship at Stanford University and by the Rackham Graduate School at the University of Michigan through a Predoctoral Fellowship. GB has been supported by grants AST-1615555 from the US National Science Foundation, and DE-SC0007901 from the US Department of Energy. DH has been supported by DOE under Contract DE-FG02-95ER40899 and NSF under contract AST-1813834. This paper has gone through internal review by the DES collaboration. JM has been supported by the Porat Fellowship at Stanford University and by the RackhamGraduate School at theUniversity of Michigan through a Predoctoral Fellowship. GB has been supported by grants AST-1615555 from the US National Science Foundation, and DE-SC0007901 from the US Department of Energy. DH has been supported by DOE under Contract DE-FG02-95ER40899 and NSF under contract AST-1813834. The analysis and framing of this paper benefited from discussions at the 'Blind Analysis in High Stakes Survey Science:When, Why, and How?' workshop12 held in 2017 March at KIPAC/SLAC. We thank the organizers and attendees of that workshop for sharing their insight and experiences. The analysis made use of the software tools scipy (Jones et al. 2001), numpy (Oliphant 2006), matplotlib (Hunter 2007), getdist (Lewis 2019), MULTINEST (Feroz & Hobson 2008; Feroz et al. 2009, 2013), cosmosis (Zuntz et al. 2015), and cosmolike (Krause & Eifler 2017). It was supported in part through computational resources and services provided by Advanced Research Computing at the National Science Foundation, Ann Arbor; the National Energy Research Scientific Computing Center (NERSC), a US Department of Energy Office of Science User Facility operated under Contract No. DE-AC02-05CH11231; and the Sherlock cluster, supported by Stanford University and the Stanford Research Computing Center.We would like to thank all of these facilities for providing computational resources and support that contributed to these research results. Funding for the DES Projects has been provided by the US Department of Energy, the US National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the National Center for SupercomputingApplications at theUniversity of Illinois at Urbana-Champaign, the Kavli Institute for Cosmological Physics at the University of Chicago, Financiadora de Estudos e Projetos, Fundacao Carlos Chagas Filho de Amparo a Pesquisa do Estado do Rio de Janeiro, Conselho Nacional de Desenvolvimento Cientifico e Tecnologico and the Ministerio da Ciencia e Tecnologia, and the Collaborating Institutions in the Dark Energy Survey. The Collaborating Institutions are Argonne National Laboratories, the University of Cambridge, Centro de Investigaciones Energeticas, Medioambientales y Tecnologicas-Madrid, the University of Chicago, University College London, DES-Brazil, Fermilab, the University of Edinburgh, the University of Illinois at Urbana- Champaign, the Institut de Ciencies de l'Espai (IEEC/CSIC), the Institut de Fisica d'Altes Energies, the Lawrence Berkeley National Laboratory, the University of Michigan, the National Optical Astronomy Observatory, the Ohio State University, the University of Pennsylvania, the University of Portsmouth, and the University of Sussex. Funding for the DES Projects has been provided by the US Department of Energy, the US National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign, the Kavli Institute for Cosmological Physics at the University of Chicago, Financiadora de Estudos e Projetos, Fundacao Carlos Chagas Filho de Amparo a Pesquisa do Estado do Rio de Janeiro , Conselho Nacional de Desenvolvimento Cientifico e Tecnologico and the Ministerio da Ciencia e Tecnologia, and the Collaborating Institutions in the Dark Energy Survey.

Funders	Funder number
Collaborating Institutions are Argonne National Laboratories
Collaborating Institutions in the Dark Energy Survey
Conselho Nacional de Desenvolvimento Cientifico e Tecnologico
Institut de Ciencies de l'Espai
Institut de Fisica d'Altes Energies
KIPAC
Kavli Institute for Cosmological Physics at the University of Chicago
Rackham Graduate School at the University of Michigan	AST-1615555
RackhamGraduate School at theUniversity of Michigan
Science and Technology Facilities Council of the United Kingdom
Stanford Research Computing Center
National Science Foundation	AST-1813834, DE-SC0007901
U.S. Department of Energy	DE-AC02-05CH11231, DE-FG02-95ER40899
Directorate for Mathematical and Physical Sciences	1813834
University of Illinois at Urbana-Champaign
Stanford University
Lawrence Berkeley National Laboratory
University of Pennsylvania
Ohio State University
University of Chicago
University of Michigan
University of Portsmouth
SLAC National Accelerator Laboratory
University of Cambridge
University College London
University of Sussex
University of Edinburgh
Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro
Financiadora de Estudos e Projetos
Ministério da Ciência e Tecnologia
Ministry of Education and Science of Ukraine
Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas

Keywords

Cosmology: observations; large-scale structure of Universe
Methods: data analysis
Methods: numerical
Methods: statistical

Access to Document

10.1093/MNRAS/STAA965

Cite this

Muir, J., Bernstein, G. M., Huterer, D., Elsner, F., Krause, E., Roodman, A., Allam, S., Annis, J., Avila, S., Bechtol, K., Bertin, E., Brooks, D., Buckley-Geer, E., Burke, D. L., Carnero Rosell, A., Carrasco Kind, M., Carretero, J., Cawthon, R., Costanzi, M., ... Zuntz, J. (2020). Blinding multiprobe cosmological experiments. Monthly Notices of the Royal Astronomical Society, 494(3), 4454-4470. https://doi.org/10.1093/MNRAS/STAA965

@article{928603c822e8493b91caa6de4fc16b24,

title = "Blinding multiprobe cosmological experiments",

abstract = "The goal of blinding is to hide an experiment's critical results - here the inferred cosmological parameters - until all decisions affecting its analysis have been finalized. This is especially important in the current era of precision cosmology,when the results of any newexperiment are closely scrutinized for consistency or tension with previous results. In analyses that combine multiple observational probes, like the combination of galaxy clustering and weak lensing in the Dark Energy Survey (DES), it is challenging to blind the results while retaining the ability to check for (in)consistency between different parts of the data. We propose a simple new blinding transformation, which works by modifying the summary statistics that are input to parameter estimation, such as two-point correlation functions. The transformation shifts the measured statistics to new values that are consistent with (blindly) shifted cosmological parameters while preserving internal (in)consistency. We apply the blinding transformation to simulated data for the projected DES Year 3 galaxy clustering and weak lensing analysis, demonstrating that practical blinding is achieved without significant perturbation of internalconsistency checks, asmeasured here by degradation of the χ2 between the data and best-fitting model. Our blinding method's performance is expected to improve as experiments evolve to higher precision and accuracy.",

keywords = "Cosmology: observations; large-scale structure of Universe, Methods: data analysis, Methods: numerical, Methods: statistical",

author = "J. Muir and Bernstein, {G. M.} and D. Huterer and F. Elsner and E. Krause and A. Roodman and S. Allam and J. Annis and S. Avila and K. Bechtol and E. Bertin and D. Brooks and E. Buckley-Geer and Burke, {D. L.} and {Carnero Rosell}, A. and {Carrasco Kind}, M. and J. Carretero and R. Cawthon and M. Costanzi and {Da Costa}, {L. N.} and {De Vicente}, J. and S. Desai and Dietrich, {J. P.} and P. Doel and Eifler, {T. F.} and S. Everett and P. Fosalba and J. Frieman and J. Garc{\'i}a-Bellido and Gerdes, {D. W.} and D. Gruen and Gruendl, {R. A.} and J. Gschwend and Hartley, {W. G.} and Hollowood, {D. L.} and James, {D. J.} and M. Jarvis and K. Kuehn and N. Kuropatkin and O. Lahav and M. March and Marshall, {J. L.} and P. Melchior and F. Menanteau and R. Miquel and Ogando, {R. L.C.} and A. Palmese and F. Paz-Chinch{\'o}n and Plazas, {A. A.} and Romer, {A. K.} and E. Sanchez and V. Scarpine and M. Schubnell and S. Serrano and I. Sevilla-Noarbe and M. Smith and E. Suchyta and G. Tarle and D. Thomas and Troxel, {M. A.} and Walker, {A. R.} and J. Weller and W. Wester and J. Zuntz",

note = "Publisher Copyright: {\textcopyright} 2020 The Author(s).",

year = "2020",

doi = "10.1093/MNRAS/STAA965",

language = "English",

volume = "494",

pages = "4454--4470",

journal = "Monthly Notices of the Royal Astronomical Society",

issn = "0035-8711",

publisher = "Oxford University Press",

number = "3",

}

Muir, J, Bernstein, GM, Huterer, D, Elsner, F, Krause, E, Roodman, A, Allam, S, Annis, J, Avila, S, Bechtol, K, Bertin, E, Brooks, D, Buckley-Geer, E, Burke, DL, Carnero Rosell, A, Carrasco Kind, M, Carretero, J, Cawthon, R, Costanzi, M, Da Costa, LN, De Vicente, J, Desai, S, Dietrich, JP, Doel, P, Eifler, TF, Everett, S, Fosalba, P, Frieman, J, García-Bellido, J, Gerdes, DW, Gruen, D, Gruendl, RA, Gschwend, J, Hartley, WG, Hollowood, DL, James, DJ, Jarvis, M, Kuehn, K, Kuropatkin, N, Lahav, O, March, M, Marshall, JL, Melchior, P, Menanteau, F, Miquel, R, Ogando, RLC, Palmese, A, Paz-Chinchón, F, Plazas, AA, Romer, AK, Sanchez, E, Scarpine, V, Schubnell, M, Serrano, S, Sevilla-Noarbe, I, Smith, M, Suchyta, E, Tarle, G, Thomas, D, Troxel, MA, Walker, AR, Weller, J, Wester, W & Zuntz, J 2020, 'Blinding multiprobe cosmological experiments', Monthly Notices of the Royal Astronomical Society, vol. 494, no. 3, pp. 4454-4470. https://doi.org/10.1093/MNRAS/STAA965

TY - JOUR

T1 - Blinding multiprobe cosmological experiments

AU - Muir, J.

AU - Bernstein, G. M.

AU - Huterer, D.

AU - Elsner, F.

AU - Krause, E.

AU - Roodman, A.

AU - Allam, S.

AU - Annis, J.

AU - Avila, S.

AU - Bechtol, K.

AU - Bertin, E.

AU - Brooks, D.

AU - Buckley-Geer, E.

AU - Burke, D. L.

AU - Carnero Rosell, A.

AU - Carrasco Kind, M.

AU - Carretero, J.

AU - Cawthon, R.

AU - Costanzi, M.

AU - Da Costa, L. N.

AU - De Vicente, J.

AU - Desai, S.

AU - Dietrich, J. P.

AU - Doel, P.

AU - Eifler, T. F.

AU - Everett, S.

AU - Fosalba, P.

AU - Frieman, J.

AU - García-Bellido, J.

AU - Gerdes, D. W.

AU - Gruen, D.

AU - Gruendl, R. A.

AU - Gschwend, J.

AU - Hartley, W. G.

AU - Hollowood, D. L.

AU - James, D. J.

AU - Jarvis, M.

AU - Kuehn, K.

AU - Kuropatkin, N.

AU - Lahav, O.

AU - March, M.

AU - Marshall, J. L.

AU - Melchior, P.

AU - Menanteau, F.

AU - Miquel, R.

AU - Ogando, R. L.C.

AU - Palmese, A.

AU - Paz-Chinchón, F.

AU - Plazas, A. A.

AU - Romer, A. K.

AU - Sanchez, E.

AU - Scarpine, V.

AU - Schubnell, M.

AU - Serrano, S.

AU - Sevilla-Noarbe, I.

AU - Smith, M.

AU - Suchyta, E.

AU - Tarle, G.

AU - Thomas, D.

AU - Troxel, M. A.

AU - Walker, A. R.

AU - Weller, J.

AU - Wester, W.

AU - Zuntz, J.

PY - 2020

Y1 - 2020

N2 - The goal of blinding is to hide an experiment's critical results - here the inferred cosmological parameters - until all decisions affecting its analysis have been finalized. This is especially important in the current era of precision cosmology,when the results of any newexperiment are closely scrutinized for consistency or tension with previous results. In analyses that combine multiple observational probes, like the combination of galaxy clustering and weak lensing in the Dark Energy Survey (DES), it is challenging to blind the results while retaining the ability to check for (in)consistency between different parts of the data. We propose a simple new blinding transformation, which works by modifying the summary statistics that are input to parameter estimation, such as two-point correlation functions. The transformation shifts the measured statistics to new values that are consistent with (blindly) shifted cosmological parameters while preserving internal (in)consistency. We apply the blinding transformation to simulated data for the projected DES Year 3 galaxy clustering and weak lensing analysis, demonstrating that practical blinding is achieved without significant perturbation of internalconsistency checks, asmeasured here by degradation of the χ2 between the data and best-fitting model. Our blinding method's performance is expected to improve as experiments evolve to higher precision and accuracy.

AB - The goal of blinding is to hide an experiment's critical results - here the inferred cosmological parameters - until all decisions affecting its analysis have been finalized. This is especially important in the current era of precision cosmology,when the results of any newexperiment are closely scrutinized for consistency or tension with previous results. In analyses that combine multiple observational probes, like the combination of galaxy clustering and weak lensing in the Dark Energy Survey (DES), it is challenging to blind the results while retaining the ability to check for (in)consistency between different parts of the data. We propose a simple new blinding transformation, which works by modifying the summary statistics that are input to parameter estimation, such as two-point correlation functions. The transformation shifts the measured statistics to new values that are consistent with (blindly) shifted cosmological parameters while preserving internal (in)consistency. We apply the blinding transformation to simulated data for the projected DES Year 3 galaxy clustering and weak lensing analysis, demonstrating that practical blinding is achieved without significant perturbation of internalconsistency checks, asmeasured here by degradation of the χ2 between the data and best-fitting model. Our blinding method's performance is expected to improve as experiments evolve to higher precision and accuracy.

KW - Cosmology: observations; large-scale structure of Universe

KW - Methods: data analysis

KW - Methods: numerical

KW - Methods: statistical

UR - http://www.scopus.com/inward/record.url?scp=85092577513&partnerID=8YFLogxK

U2 - 10.1093/MNRAS/STAA965

DO - 10.1093/MNRAS/STAA965

M3 - Article

AN - SCOPUS:85092577513

SN - 0035-8711

VL - 494

SP - 4454

EP - 4470

JO - Monthly Notices of the Royal Astronomical Society

JF - Monthly Notices of the Royal Astronomical Society

IS - 3

ER -

Blinding multiprobe cosmological experiments

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this