Ten questions concerning reinforcement learning for building energy management

Zoltan Nagy; Gregor Henze; Sourav Dey; Javier Arroyo; Lieve Helsen; Xiangyu Zhang; Bingqing Chen; Kadir Amasyali; Kuldeep Kurte; Ahmed Zamzam; Helia Zandi; Ján Drgoňa; Matias Quintana; Steven McCullogh; June Young Park; Han Li; Tianzhen Hong; Silvio Brandi; Giuseppe Pinto; Alfonso Capozzoli; Draguna Vrabie; Mario Bergés; Kingsley Nweye; Thibault Marzullo; Andrey Bernstein

doi:10.1016/j.buildenv.2023.110435

Ten questions concerning reinforcement learning for building energy management

Zoltan Nagy, Gregor Henze, Sourav Dey, Javier Arroyo, Lieve Helsen, Xiangyu Zhang, Bingqing Chen, Kadir Amasyali, Kuldeep Kurte, Ahmed Zamzam, Helia Zandi, Ján Drgoňa, Matias Quintana, Steven McCullogh, June Young Park, Han Li, Tianzhen Hong, Silvio Brandi, Giuseppe Pinto, Alfonso CapozzoliDraguna Vrabie, Mario Bergés, Kingsley Nweye, Thibault Marzullo, Andrey Bernstein

Computational Systems Engineering and Cy

Research output: Contribution to journal › Article › peer-review

30 Scopus citations

Abstract

As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

Original language	English
Article number	110435
Journal	Building and Environment
Volume	241
DOIs	https://doi.org/10.1016/j.buildenv.2023.110435
State	Published - Aug 1 2023

Funding

PNNL authors are supported by the U.S. Department of Energy, United States , through the Energy Efficiency and Renewable Energy, Building Technologies Office under the “Advancing Market-Ready Building Energy Management by Cost-Effective Differentiable Predictive Control” project. PNNL is a multi-program national laboratory operated for the U.S. Department of Energy (DOE), United States by Battelle Memorial Institute under Contract No. DE-AC05-76RL0-1830 . LBNL authors are supported by the U.S. Department of Energy, United States , through the Energy Efficiency and Renewable Energy, Building Technologies Office under Contract No. DE-AC02-05CH11231. The National Renewable Energy Laboratory (NREL) authors are supported by the U.S. Department of Energy, United States , through the Energy Efficiency and Renewable Energy, Building Technologies Office. NREL is operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract DE-AC36-08GO28308. The NUS authors are funded by the NUS-based Singapore MOE Tier 1 Grant titled Ecological Momentary Assessment (EMA) for Built Environment Research ( A-0008301-01-00 ). Part of this research was conducted at the Future Cities Lab Global at Singapore-ETH Centre. Future Cities Lab Global is supported and funded by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme and ETH Zurich (ETHZ), with additional contributions from the National University of Singapore (NUS) , Nanyang Technological University (NTU), Singapore , Singapore and the Singapore University of Technology and Design (SUTD) . The KU Leuven authors acknowledge the financial support through the TECHPED - C2 project (C24M/21/021). The TECHPED project investigates TECHnically feasible and effective solutions for Positive Energy Districts. ORNL staff were supported by the US Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725 .

Funders	Funder number
Energy Efficiency and Renewable Energy, Building Technologies Office
TECHPED	C24M/21/021
US Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office	DE-AC05-00OR22725
U.S. Department of Energy
Battelle	DE-AC05-76RL0-1830, DE-AC02-05CH11231
National Renewable Energy Laboratory	DE-AC36-08GO28308
National University of Singapore
National Research Foundation Singapore
Ministry of Education - Singapore	A-0008301-01-00
Nanyang Technological University
Eidgenössische Technische Hochschule Zürich
KU Leuven
Singapore University of Technology and Design

Keywords

Open AI Gym

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.buildenv.2023.110435

Cite this

Nagy, Z., Henze, G., Dey, S., Arroyo, J., Helsen, L., Zhang, X., Chen, B., Amasyali, K., Kurte, K., Zamzam, A., Zandi, H., Drgoňa, J., Quintana, M., McCullogh, S., Park, J. Y., Li, H., Hong, T., Brandi, S., Pinto, G., ... Bernstein, A. (2023). Ten questions concerning reinforcement learning for building energy management. Building and Environment, 241, Article 110435. https://doi.org/10.1016/j.buildenv.2023.110435

@article{4d521b3b0218472bbfe6d619752bb4ff,

title = "Ten questions concerning reinforcement learning for building energy management",

abstract = "As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.",

keywords = "Open AI Gym",

author = "Zoltan Nagy and Gregor Henze and Sourav Dey and Javier Arroyo and Lieve Helsen and Xiangyu Zhang and Bingqing Chen and Kadir Amasyali and Kuldeep Kurte and Ahmed Zamzam and Helia Zandi and J{\'a}n Drgo{\v n}a and Matias Quintana and Steven McCullogh and Park, {June Young} and Han Li and Tianzhen Hong and Silvio Brandi and Giuseppe Pinto and Alfonso Capozzoli and Draguna Vrabie and Mario Berg{\'e}s and Kingsley Nweye and Thibault Marzullo and Andrey Bernstein",

note = "Publisher Copyright: {\textcopyright} 2023 Elsevier Ltd",

year = "2023",

month = aug,

day = "1",

doi = "10.1016/j.buildenv.2023.110435",

language = "English",

volume = "241",

journal = "Building and Environment",

issn = "0360-1323",

publisher = "Elsevier",

}

Nagy, Z, Henze, G, Dey, S, Arroyo, J, Helsen, L, Zhang, X, Chen, B, Amasyali, K, Kurte, K, Zamzam, A, Zandi, H, Drgoňa, J, Quintana, M, McCullogh, S, Park, JY, Li, H, Hong, T, Brandi, S, Pinto, G, Capozzoli, A, Vrabie, D, Bergés, M, Nweye, K, Marzullo, T & Bernstein, A 2023, 'Ten questions concerning reinforcement learning for building energy management', Building and Environment, vol. 241, 110435. https://doi.org/10.1016/j.buildenv.2023.110435

TY - JOUR

T1 - Ten questions concerning reinforcement learning for building energy management

AU - Nagy, Zoltan

AU - Henze, Gregor

AU - Dey, Sourav

AU - Arroyo, Javier

AU - Helsen, Lieve

AU - Zhang, Xiangyu

AU - Chen, Bingqing

AU - Amasyali, Kadir

AU - Kurte, Kuldeep

AU - Zamzam, Ahmed

AU - Zandi, Helia

AU - Drgoňa, Ján

AU - Quintana, Matias

AU - McCullogh, Steven

AU - Park, June Young

AU - Li, Han

AU - Hong, Tianzhen

AU - Brandi, Silvio

AU - Pinto, Giuseppe

AU - Capozzoli, Alfonso

AU - Vrabie, Draguna

AU - Bergés, Mario

AU - Nweye, Kingsley

AU - Marzullo, Thibault

AU - Bernstein, Andrey

PY - 2023/8/1

Y1 - 2023/8/1

N2 - As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

AB - As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

KW - Open AI Gym

UR - http://www.scopus.com/inward/record.url?scp=85161304971&partnerID=8YFLogxK

U2 - 10.1016/j.buildenv.2023.110435

DO - 10.1016/j.buildenv.2023.110435

M3 - Article

AN - SCOPUS:85161304971

SN - 0360-1323

VL - 241

JO - Building and Environment

JF - Building and Environment

M1 - 110435

ER -

Ten questions concerning reinforcement learning for building energy management

Abstract

Funding

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this