Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment

Kadir Amasyali; Kuldeep Kurte; Helia Zandi; Jeffrey Munk

doi:10.1109/ISGT51731.2023.10066373

Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment

Kadir Amasyali, Kuldeep Kurte, Helia Zandi, Jeffrey Munk

Computational Systems Engineering and Cy

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.

Original language	English
Title of host publication	2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665453554
DOIs	https://doi.org/10.1109/ISGT51731.2023.10066373
State	Published - 2023
Event	2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023 - Washington, United States Duration: Jan 16 2023 → Jan 19 2023

Publication series

Name	2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023

Conference

Conference	2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023
Country/Territory	United States
City	Washington
Period	01/16/23 → 01/19/23

Funding

This work was funded by the U.S. Department of Energy, Energy Efficiency and Renewable Energy, Building Technology Office under contract number DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

Demand response (DR)
deep learning
electric water heaters
model-free control
reinforcement learning (RL)

Access to Document

10.1109/ISGT51731.2023.10066373

Cite this

Amasyali, K., Kurte, K., Zandi, H., & Munk, J. (2023). Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment. In 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023 (2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISGT51731.2023.10066373

Amasyali, Kadir ; Kurte, Kuldeep ; Zandi, Helia et al. / Reinforcement-learning-based Smart Water Heater Control : An Actual Deployment. 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023. Institute of Electrical and Electronics Engineers Inc., 2023. (2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023).

@inproceedings{07d644db2a79431885bed386cafb87dc,

title = "Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment",

abstract = "Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.",

keywords = "Demand response (DR), deep learning, electric water heaters, model-free control, reinforcement learning (RL)",

author = "Kadir Amasyali and Kuldeep Kurte and Helia Zandi and Jeffrey Munk",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023 ; Conference date: 16-01-2023 Through 19-01-2023",

year = "2023",

doi = "10.1109/ISGT51731.2023.10066373",

language = "English",

series = "2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023",

}

Amasyali, K, Kurte, K, Zandi, H & Munk, J 2023, Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment. in 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023. 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023, Institute of Electrical and Electronics Engineers Inc., 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023, Washington, United States, 01/16/23. https://doi.org/10.1109/ISGT51731.2023.10066373

Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment. / Amasyali, Kadir; Kurte, Kuldeep; Zandi, Helia et al.
2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023. Institute of Electrical and Electronics Engineers Inc., 2023. (2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement-learning-based Smart Water Heater Control

T2 - 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023

AU - Amasyali, Kadir

AU - Kurte, Kuldeep

AU - Zandi, Helia

AU - Munk, Jeffrey

PY - 2023

Y1 - 2023

N2 - Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.

AB - Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.

KW - Demand response (DR)

KW - deep learning

KW - electric water heaters

KW - model-free control

KW - reinforcement learning (RL)

UR - http://www.scopus.com/inward/record.url?scp=85151531835&partnerID=8YFLogxK

U2 - 10.1109/ISGT51731.2023.10066373

DO - 10.1109/ISGT51731.2023.10066373

M3 - Conference contribution

AN - SCOPUS:85151531835

T3 - 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023

BT - 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 16 January 2023 through 19 January 2023

ER -

Amasyali K, Kurte K, Zandi H, Munk J. Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment. In 2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023. Institute of Electrical and Electronics Engineers Inc. 2023. (2023 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference, ISGT 2023). doi: 10.1109/ISGT51731.2023.10066373

Reinforcement-learning-based Smart Water Heater Control: An Actual Deployment

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this