Deep reinforcement learning for residential hvac control with consideration of human occupancy

Evan McKee; Yan Du; Fangxing Li; Jeffrey Munk; Travis Johnston; Kuldeep Kurte; Olivera Kotevska; Kadir Amasyali; Helia Zandi

doi:10.1109/PESGM41954.2020.9281893

Deep reinforcement learning for residential hvac control with consideration of human occupancy

Evan McKee, Yan Du, Fangxing Li, Jeffrey Munk, Travis Johnston, Kuldeep Kurte, Olivera Kotevska, Kadir Amasyali, Helia Zandi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.

Original language	English
Title of host publication	2020 IEEE Power and Energy Society General Meeting, PESGM 2020
Publisher	IEEE Computer Society
ISBN (Electronic)	9781728155081
DOIs	https://doi.org/10.1109/PESGM41954.2020.9281893
State	Published - Aug 2 2020
Event	2020 IEEE Power and Energy Society General Meeting, PESGM 2020 - Montreal, Canada Duration: Aug 2 2020 → Aug 6 2020

Publication series

Name	IEEE Power and Energy Society General Meeting
Volume	2020-August
ISSN (Print)	1944-9925
ISSN (Electronic)	1944-9933

Conference

Conference	2020 IEEE Power and Energy Society General Meeting, PESGM 2020
Country/Territory	Canada
City	Montreal
Period	08/2/20 → 08/6/20

Funding

ACKNOWLEDGMENT This work was funded by the U.S. Department of Energy, Energy Efficiency, and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725. This work was funded by the U.S. Department of Energy, Energy Efficiency, and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725. This work would not have been possible without the continued support and sponsorship of Oak Ridge National Laboratory and its affiliates. CURENT and its affiliates, including NSF, are gratefully acknowledged.

Keywords

Automation
Demand response
Machine learning
Smart grid
Transactive control

Access to Document

10.1109/PESGM41954.2020.9281893

Cite this

McKee, E., Du, Y., Li, F., Munk, J., Johnston, T., Kurte, K., Kotevska, O., Amasyali, K., & Zandi, H. (2020). Deep reinforcement learning for residential hvac control with consideration of human occupancy. In 2020 IEEE Power and Energy Society General Meeting, PESGM 2020 Article 9281893 (IEEE Power and Energy Society General Meeting; Vol. 2020-August). IEEE Computer Society. https://doi.org/10.1109/PESGM41954.2020.9281893

@inproceedings{40a3ba92072d48279a3fe4fb8e66af7a,

title = "Deep reinforcement learning for residential hvac control with consideration of human occupancy",

abstract = "The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.",

keywords = "Automation, Demand response, Machine learning, Smart grid, Transactive control",

author = "Evan McKee and Yan Du and Fangxing Li and Jeffrey Munk and Travis Johnston and Kuldeep Kurte and Olivera Kotevska and Kadir Amasyali and Helia Zandi",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE Power and Energy Society General Meeting, PESGM 2020 ; Conference date: 02-08-2020 Through 06-08-2020",

year = "2020",

month = aug,

day = "2",

doi = "10.1109/PESGM41954.2020.9281893",

language = "English",

series = "IEEE Power and Energy Society General Meeting",

publisher = "IEEE Computer Society",

booktitle = "2020 IEEE Power and Energy Society General Meeting, PESGM 2020",

}

McKee, E, Du, Y, Li, F, Munk, J, Johnston, T, Kurte, K, Kotevska, O, Amasyali, K & Zandi, H 2020, Deep reinforcement learning for residential hvac control with consideration of human occupancy. in 2020 IEEE Power and Energy Society General Meeting, PESGM 2020., 9281893, IEEE Power and Energy Society General Meeting, vol. 2020-August, IEEE Computer Society, 2020 IEEE Power and Energy Society General Meeting, PESGM 2020, Montreal, Canada, 08/2/20. https://doi.org/10.1109/PESGM41954.2020.9281893

Deep reinforcement learning for residential hvac control with consideration of human occupancy. / McKee, Evan; Du, Yan; Li, Fangxing et al.
2020 IEEE Power and Energy Society General Meeting, PESGM 2020. IEEE Computer Society, 2020. 9281893 (IEEE Power and Energy Society General Meeting; Vol. 2020-August).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Deep reinforcement learning for residential hvac control with consideration of human occupancy

AU - McKee, Evan

AU - Du, Yan

AU - Li, Fangxing

AU - Munk, Jeffrey

AU - Johnston, Travis

AU - Kurte, Kuldeep

AU - Kotevska, Olivera

AU - Amasyali, Kadir

AU - Zandi, Helia

PY - 2020/8/2

Y1 - 2020/8/2

N2 - The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.

AB - The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.

KW - Automation

KW - Demand response

KW - Machine learning

KW - Smart grid

KW - Transactive control

UR - http://www.scopus.com/inward/record.url?scp=85099126330&partnerID=8YFLogxK

U2 - 10.1109/PESGM41954.2020.9281893

DO - 10.1109/PESGM41954.2020.9281893

M3 - Conference contribution

AN - SCOPUS:85099126330

T3 - IEEE Power and Energy Society General Meeting

BT - 2020 IEEE Power and Energy Society General Meeting, PESGM 2020

PB - IEEE Computer Society

T2 - 2020 IEEE Power and Energy Society General Meeting, PESGM 2020

Y2 - 2 August 2020 through 6 August 2020

ER -

Deep reinforcement learning for residential hvac control with consideration of human occupancy

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this