Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control

Kuldeep Kurte; Jeffrey Munk; Kadir Amasyali; Olivera Kotevska; Borui Cui; Teja Kuruganti; Helia Zandi

doi:10.1145/3427773.3427866

Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control

Kuldeep Kurte, Jeffrey Munk, Kadir Amasyali, Olivera Kotevska, Borui Cui, Teja Kuruganti, Helia Zandi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Recently, deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) has gained a lot of attention due to DRL's ability to optimally control HVAC for minimizing operational cost while maintaining resident's comfort. The success of such DRL-based techniques largely depends on the articulation of the problem in terms of states, actions, and reward function. Inclusion of the electricity pricing information in the problem formulation can play an important role in saving the cost of HVAC operation. However, less attention has been given in the literature on formulating well-crafted state features based on electricity pricing. In this work, we propose an approach for training the DRL model with a specific focus on feature engineering based on electricity pricing. During training, we generate random but sufficiently realistic electricity price signals so that the pre-trained DRL model is robust and adaptive to the dynamic and variable electricity prices. The validation results are encouraging and show the potential of ≈12%-15% savings in the one day cost of HVAC operation, proving the usefulness of including electricity pricing related features as state features.

Original language	English
Title of host publication	RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities
Publisher	Association for Computing Machinery, Inc
Pages	6-10
Number of pages	5
ISBN (Electronic)	9781450381932
DOIs	https://doi.org/10.1145/3427773.3427866
State	Published - Nov 17 2020
Event	1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, RLEM 2020 - Virtual, Online, Japan Duration: Nov 17 2020 → …

Publication series

Name	RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities

Conference

Conference	1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, RLEM 2020
Country/Territory	Japan
City	Virtual, Online
Period	11/17/20 → …

Funding

This work was funded by the U.S. Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). This work was funded by the U.S. Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05- 00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

building energy
deep reinforcement learning
demand response
intelligent HVAC control

Access to Document

10.1145/3427773.3427866

Cite this

Kurte, K., Munk, J., Amasyali, K., Kotevska, O., Cui, B., Kuruganti, T., & Zandi, H. (2020). Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control. In RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities (pp. 6-10). (RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities). Association for Computing Machinery, Inc. https://doi.org/10.1145/3427773.3427866

Kurte, Kuldeep ; Munk, Jeffrey ; Amasyali, Kadir et al. / Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control. RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities. Association for Computing Machinery, Inc, 2020. pp. 6-10 (RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities).

@inproceedings{b071b090c31c42ec9b1156c755f857e0,

title = "Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control",

abstract = "Recently, deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) has gained a lot of attention due to DRL's ability to optimally control HVAC for minimizing operational cost while maintaining resident's comfort. The success of such DRL-based techniques largely depends on the articulation of the problem in terms of states, actions, and reward function. Inclusion of the electricity pricing information in the problem formulation can play an important role in saving the cost of HVAC operation. However, less attention has been given in the literature on formulating well-crafted state features based on electricity pricing. In this work, we propose an approach for training the DRL model with a specific focus on feature engineering based on electricity pricing. During training, we generate random but sufficiently realistic electricity price signals so that the pre-trained DRL model is robust and adaptive to the dynamic and variable electricity prices. The validation results are encouraging and show the potential of ≈12%-15% savings in the one day cost of HVAC operation, proving the usefulness of including electricity pricing related features as state features.",

keywords = "building energy, deep reinforcement learning, demand response, intelligent HVAC control",

author = "Kuldeep Kurte and Jeffrey Munk and Kadir Amasyali and Olivera Kotevska and Borui Cui and Teja Kuruganti and Helia Zandi",

note = "Publisher Copyright: {\textcopyright} 2020 ACM.; 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, RLEM 2020 ; Conference date: 17-11-2020",

year = "2020",

month = nov,

day = "17",

doi = "10.1145/3427773.3427866",

language = "English",

series = "RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities",

publisher = "Association for Computing Machinery, Inc",

pages = "6--10",

booktitle = "RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities",

}

Kurte, K, Munk, J, Amasyali, K, Kotevska, O , Cui, B , Kuruganti, T & Zandi, H 2020, Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control. in RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities. RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, Association for Computing Machinery, Inc, pp. 6-10, 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, RLEM 2020, Virtual, Online, Japan, 11/17/20. https://doi.org/10.1145/3427773.3427866

Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control. / Kurte, Kuldeep; Munk, Jeffrey; Amasyali, Kadir et al.
RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities. Association for Computing Machinery, Inc, 2020. p. 6-10 (RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control

AU - Kurte, Kuldeep

AU - Munk, Jeffrey

AU - Amasyali, Kadir

AU - Kotevska, Olivera

AU - Cui, Borui

AU - Kuruganti, Teja

AU - Zandi, Helia

PY - 2020/11/17

Y1 - 2020/11/17

N2 - Recently, deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) has gained a lot of attention due to DRL's ability to optimally control HVAC for minimizing operational cost while maintaining resident's comfort. The success of such DRL-based techniques largely depends on the articulation of the problem in terms of states, actions, and reward function. Inclusion of the electricity pricing information in the problem formulation can play an important role in saving the cost of HVAC operation. However, less attention has been given in the literature on formulating well-crafted state features based on electricity pricing. In this work, we propose an approach for training the DRL model with a specific focus on feature engineering based on electricity pricing. During training, we generate random but sufficiently realistic electricity price signals so that the pre-trained DRL model is robust and adaptive to the dynamic and variable electricity prices. The validation results are encouraging and show the potential of ≈12%-15% savings in the one day cost of HVAC operation, proving the usefulness of including electricity pricing related features as state features.

AB - Recently, deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) has gained a lot of attention due to DRL's ability to optimally control HVAC for minimizing operational cost while maintaining resident's comfort. The success of such DRL-based techniques largely depends on the articulation of the problem in terms of states, actions, and reward function. Inclusion of the electricity pricing information in the problem formulation can play an important role in saving the cost of HVAC operation. However, less attention has been given in the literature on formulating well-crafted state features based on electricity pricing. In this work, we propose an approach for training the DRL model with a specific focus on feature engineering based on electricity pricing. During training, we generate random but sufficiently realistic electricity price signals so that the pre-trained DRL model is robust and adaptive to the dynamic and variable electricity prices. The validation results are encouraging and show the potential of ≈12%-15% savings in the one day cost of HVAC operation, proving the usefulness of including electricity pricing related features as state features.

KW - building energy

KW - deep reinforcement learning

KW - demand response

KW - intelligent HVAC control

UR - http://www.scopus.com/inward/record.url?scp=85097538298&partnerID=8YFLogxK

U2 - 10.1145/3427773.3427866

DO - 10.1145/3427773.3427866

M3 - Conference contribution

AN - SCOPUS:85097538298

T3 - RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities

SP - 6

EP - 10

BT - RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities

PB - Association for Computing Machinery, Inc

T2 - 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, RLEM 2020

Y2 - 17 November 2020

ER -

Kurte K, Munk J, Amasyali K, Kotevska O , Cui B , Kuruganti T et al. Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control. In RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities. Association for Computing Machinery, Inc. 2020. p. 6-10. (RLEM 2020 - Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings and Cities). doi: 10.1145/3427773.3427866

Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this