Ten questions concerning reinforcement learning for building energy management

Zoltan Nagy, Gregor Henze, Sourav Dey, Javier Arroyo, Lieve Helsen, Xiangyu Zhang, Bingqing Chen, Kadir Amasyali, Kuldeep Kurte, Ahmed Zamzam, Helia Zandi, Ján Drgoňa, Matias Quintana, Steven McCullogh, June Young Park, Han Li, Tianzhen Hong, Silvio Brandi, Giuseppe Pinto, Alfonso CapozzoliDraguna Vrabie, Mario Bergés, Kingsley Nweye, Thibault Marzullo, Andrey Bernstein

Research output: Contribution to journalArticlepeer-review

30 Scopus citations

Abstract

As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

Original languageEnglish
Article number110435
JournalBuilding and Environment
Volume241
DOIs
StatePublished - Aug 1 2023

Funding

PNNL authors are supported by the U.S. Department of Energy, United States , through the Energy Efficiency and Renewable Energy, Building Technologies Office under the “Advancing Market-Ready Building Energy Management by Cost-Effective Differentiable Predictive Control” project. PNNL is a multi-program national laboratory operated for the U.S. Department of Energy (DOE), United States by Battelle Memorial Institute under Contract No. DE-AC05-76RL0-1830 . LBNL authors are supported by the U.S. Department of Energy, United States , through the Energy Efficiency and Renewable Energy, Building Technologies Office under Contract No. DE-AC02-05CH11231. The National Renewable Energy Laboratory (NREL) authors are supported by the U.S. Department of Energy, United States , through the Energy Efficiency and Renewable Energy, Building Technologies Office. NREL is operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract DE-AC36-08GO28308. The NUS authors are funded by the NUS-based Singapore MOE Tier 1 Grant titled Ecological Momentary Assessment (EMA) for Built Environment Research ( A-0008301-01-00 ). Part of this research was conducted at the Future Cities Lab Global at Singapore-ETH Centre. Future Cities Lab Global is supported and funded by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme and ETH Zurich (ETHZ), with additional contributions from the National University of Singapore (NUS) , Nanyang Technological University (NTU), Singapore , Singapore and the Singapore University of Technology and Design (SUTD) . The KU Leuven authors acknowledge the financial support through the TECHPED - C2 project (C24M/21/021). The TECHPED project investigates TECHnically feasible and effective solutions for Positive Energy Districts. ORNL staff were supported by the US Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725 .

FundersFunder number
Energy Efficiency and Renewable Energy, Building Technologies Office
TECHPEDC24M/21/021
US Department of Energy, Energy Efficiency and Renewable Energy, Building Technology OfficeDE-AC05-00OR22725
U.S. Department of Energy
BattelleDE-AC05-76RL0-1830, DE-AC02-05CH11231
National Renewable Energy LaboratoryDE-AC36-08GO28308
National University of Singapore
National Research Foundation Singapore
Ministry of Education - SingaporeA-0008301-01-00
Nanyang Technological University
Eidgenössische Technische Hochschule Zürich
KU Leuven
Singapore University of Technology and Design

    Keywords

    • Open AI Gym

    Fingerprint

    Dive into the research topics of 'Ten questions concerning reinforcement learning for building energy management'. Together they form a unique fingerprint.

    Cite this