Comparative analysis of model-free and model-based HVAC control for residential demand response

Kuldeep Kurte; Kadir Amasyali; Jeffrey Munk; Helia Zandi

doi:10.1145/3486611.3488727

Comparative analysis of model-free and model-based HVAC control for residential demand response

Kuldeep Kurte, Kadir Amasyali, Jeffrey Munk, Helia Zandi

Computational Systems Engineering and Cy

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Scopus citations

Abstract

In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.

Original language	English
Title of host publication	BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments
Publisher	Association for Computing Machinery, Inc
Pages	309-313
Number of pages	5
ISBN (Electronic)	9781450391146
DOIs	https://doi.org/10.1145/3486611.3488727
State	Published - Nov 17 2021
Event	8th ACM International Conference on Systems for Energy-Efficient Built Environments, BuildSys 2021 - Virtual, Online, Portugal Duration: Nov 17 2021 → Nov 18 2021

Publication series

Name	BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments

Conference

Conference	8th ACM International Conference on Systems for Energy-Efficient Built Environments, BuildSys 2021
Country/Territory	Portugal
City	Virtual, Online
Period	11/17/21 → 11/18/21

Funding

This work was funded by the U.S. Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

building energy
deep reinforcement learning
demand response
intelligent HVAC control
model predictive control

Access to Document

10.1145/3486611.3488727

Cite this

Kurte, K., Amasyali, K., Munk, J., & Zandi, H. (2021). Comparative analysis of model-free and model-based HVAC control for residential demand response. In BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments (pp. 309-313). (BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments). Association for Computing Machinery, Inc. https://doi.org/10.1145/3486611.3488727

Kurte, Kuldeep ; Amasyali, Kadir ; Munk, Jeffrey et al. / Comparative analysis of model-free and model-based HVAC control for residential demand response. BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments. Association for Computing Machinery, Inc, 2021. pp. 309-313 (BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments).

@inproceedings{6eb4ee0a918049708533f126095085b0,

title = "Comparative analysis of model-free and model-based HVAC control for residential demand response",

abstract = "In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.",

keywords = "building energy, deep reinforcement learning, demand response, intelligent HVAC control, model predictive control",

author = "Kuldeep Kurte and Kadir Amasyali and Jeffrey Munk and Helia Zandi",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 8th ACM International Conference on Systems for Energy-Efficient Built Environments, BuildSys 2021 ; Conference date: 17-11-2021 Through 18-11-2021",

year = "2021",

month = nov,

day = "17",

doi = "10.1145/3486611.3488727",

language = "English",

series = "BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments",

publisher = "Association for Computing Machinery, Inc",

pages = "309--313",

booktitle = "BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments",

}

Kurte, K, Amasyali, K, Munk, J & Zandi, H 2021, Comparative analysis of model-free and model-based HVAC control for residential demand response. in BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments. BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments, Association for Computing Machinery, Inc, pp. 309-313, 8th ACM International Conference on Systems for Energy-Efficient Built Environments, BuildSys 2021, Virtual, Online, Portugal, 11/17/21. https://doi.org/10.1145/3486611.3488727

Comparative analysis of model-free and model-based HVAC control for residential demand response. / Kurte, Kuldeep; Amasyali, Kadir; Munk, Jeffrey et al.
BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments. Association for Computing Machinery, Inc, 2021. p. 309-313 (BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Comparative analysis of model-free and model-based HVAC control for residential demand response

AU - Kurte, Kuldeep

AU - Amasyali, Kadir

AU - Munk, Jeffrey

AU - Zandi, Helia

PY - 2021/11/17

Y1 - 2021/11/17

N2 - In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.

AB - In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.

KW - building energy

KW - deep reinforcement learning

KW - demand response

KW - intelligent HVAC control

KW - model predictive control

UR - http://www.scopus.com/inward/record.url?scp=85120995796&partnerID=8YFLogxK

U2 - 10.1145/3486611.3488727

DO - 10.1145/3486611.3488727

M3 - Conference contribution

AN - SCOPUS:85120995796

T3 - BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments

SP - 309

EP - 313

BT - BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments

PB - Association for Computing Machinery, Inc

T2 - 8th ACM International Conference on Systems for Energy-Efficient Built Environments, BuildSys 2021

Y2 - 17 November 2021 through 18 November 2021

ER -

Kurte K, Amasyali K, Munk J, Zandi H. Comparative analysis of model-free and model-based HVAC control for residential demand response. In BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments. Association for Computing Machinery, Inc. 2021. p. 309-313. (BuildSys 2021 - Proceedings of the 2021 ACM International Conference on Systems for Energy-Efficient Built Environments). doi: 10.1145/3486611.3488727

Comparative analysis of model-free and model-based HVAC control for residential demand response

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this