Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control

Olivera Kotevska; Jeffrey Munk; Kuldeep Kurte; Yan Du; Kadir Amasyali; Robert W. Smith; Helia Zandi

doi:10.1109/BigData50022.2020.9377735

Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control

Olivera Kotevska, Jeffrey Munk, Kuldeep Kurte, Yan Du, Kadir Amasyali, Robert W. Smith, Helia Zandi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Scopus citations

Abstract

Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.

Original language	English
Title of host publication	Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020
Editors	Xintao Wu, Chris Jermaine, Li Xiong, Xiaohua Tony Hu, Olivera Kotevska, Siyuan Lu, Weijia Xu, Srinivas Aluru, Chengxiang Zhai, Eyhab Al-Masri, Zhiyuan Chen, Jeff Saltz
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1555-1564
Number of pages	10
ISBN (Electronic)	9781728162515
DOIs	https://doi.org/10.1109/BigData50022.2020.9377735
State	Published - Dec 10 2020
Event	8th IEEE International Conference on Big Data, Big Data 2020 - Virtual, Atlanta, United States Duration: Dec 10 2020 → Dec 13 2020

Publication series

Name	Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

Conference

Conference	8th IEEE International Conference on Big Data, Big Data 2020
Country/Territory	United States
City	Virtual, Atlanta
Period	12/10/20 → 12/13/20

Funding

ACKNOWLEDGMENT This work was funded by the U.S. Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725.

Funders	Funder number
Energy Efficiency and Renewable Energy, Building Technology Office	DE-AC05-00OR22725
U.S. Department of Energy

Keywords

decision making
deep learning
demand response
interpretability
machine learning
optimization
reinforcement learning

Access to Document

10.1109/BigData50022.2020.9377735

Cite this

Kotevska, O., Munk, J., Kurte, K., Du, Y., Amasyali, K., Smith, R. W., & Zandi, H. (2020). Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control. In X. Wu, C. Jermaine, L. Xiong, X. T. Hu, O. Kotevska, S. Lu, W. Xu, S. Aluru, C. Zhai, E. Al-Masri, Z. Chen, & J. Saltz (Eds.), Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020 (pp. 1555-1564). Article 9377735 (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData50022.2020.9377735

Kotevska, Olivera ; Munk, Jeffrey ; Kurte, Kuldeep et al. / Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control. Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020. editor / Xintao Wu ; Chris Jermaine ; Li Xiong ; Xiaohua Tony Hu ; Olivera Kotevska ; Siyuan Lu ; Weijia Xu ; Srinivas Aluru ; Chengxiang Zhai ; Eyhab Al-Masri ; Zhiyuan Chen ; Jeff Saltz. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 1555-1564 (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020).

@inproceedings{e40c493a3be34e5eb2dae5cd4d8fce9c,

title = "Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control",

abstract = "Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.",

keywords = "decision making, deep learning, demand response, interpretability, machine learning, optimization, reinforcement learning",

author = "Olivera Kotevska and Jeffrey Munk and Kuldeep Kurte and Yan Du and Kadir Amasyali and Smith, {Robert W.} and Helia Zandi",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 8th IEEE International Conference on Big Data, Big Data 2020 ; Conference date: 10-12-2020 Through 13-12-2020",

year = "2020",

month = dec,

day = "10",

doi = "10.1109/BigData50022.2020.9377735",

language = "English",

series = "Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1555--1564",

editor = "Xintao Wu and Chris Jermaine and Li Xiong and Hu, {Xiaohua Tony} and Olivera Kotevska and Siyuan Lu and Weijia Xu and Srinivas Aluru and Chengxiang Zhai and Eyhab Al-Masri and Zhiyuan Chen and Jeff Saltz",

booktitle = "Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020",

}

Kotevska, O, Munk, J, Kurte, K, Du, Y, Amasyali, K, Smith, RW & Zandi, H 2020, Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control. in X Wu, C Jermaine, L Xiong, XT Hu, O Kotevska, S Lu, W Xu, S Aluru, C Zhai, E Al-Masri, Z Chen & J Saltz (eds), Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020., 9377735, Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020, Institute of Electrical and Electronics Engineers Inc., pp. 1555-1564, 8th IEEE International Conference on Big Data, Big Data 2020, Virtual, Atlanta, United States, 12/10/20. https://doi.org/10.1109/BigData50022.2020.9377735

Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control. / Kotevska, Olivera; Munk, Jeffrey; Kurte, Kuldeep et al.
Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020. ed. / Xintao Wu; Chris Jermaine; Li Xiong; Xiaohua Tony Hu; Olivera Kotevska; Siyuan Lu; Weijia Xu; Srinivas Aluru; Chengxiang Zhai; Eyhab Al-Masri; Zhiyuan Chen; Jeff Saltz. Institute of Electrical and Electronics Engineers Inc., 2020. p. 1555-1564 9377735 (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control

AU - Kotevska, Olivera

AU - Munk, Jeffrey

AU - Kurte, Kuldeep

AU - Du, Yan

AU - Amasyali, Kadir

AU - Smith, Robert W.

AU - Zandi, Helia

PY - 2020/12/10

Y1 - 2020/12/10

N2 - Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.

AB - Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.

KW - decision making

KW - deep learning

KW - demand response

KW - interpretability

KW - machine learning

KW - optimization

KW - reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85103834346&partnerID=8YFLogxK

U2 - 10.1109/BigData50022.2020.9377735

DO - 10.1109/BigData50022.2020.9377735

M3 - Conference contribution

AN - SCOPUS:85103834346

T3 - Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

SP - 1555

EP - 1564

BT - Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

A2 - Wu, Xintao

A2 - Jermaine, Chris

A2 - Xiong, Li

A2 - Hu, Xiaohua Tony

A2 - Kotevska, Olivera

A2 - Lu, Siyuan

A2 - Xu, Weijia

A2 - Aluru, Srinivas

A2 - Zhai, Chengxiang

A2 - Al-Masri, Eyhab

A2 - Chen, Zhiyuan

A2 - Saltz, Jeff

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 8th IEEE International Conference on Big Data, Big Data 2020

Y2 - 10 December 2020 through 13 December 2020

ER -

Kotevska O, Munk J, Kurte K, Du Y, Amasyali K, Smith RW et al. Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control. In Wu X, Jermaine C, Xiong L, Hu XT, Kotevska O, Lu S, Xu W, Aluru S, Zhai C, Al-Masri E, Chen Z, Saltz J, editors, Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020. Institute of Electrical and Electronics Engineers Inc. 2020. p. 1555-1564. 9377735. (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020). doi: 10.1109/BigData50022.2020.9377735

Methodology for Interpretable Reinforcement Learning Model for HVAC Energy Control

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this