Deep reinforcement learning with online data augmentation to improve sample efficiency for intelligent HVAC control

Kuldeep Kurte, Kadir Amasyali, Jeffrey Munk, Helia Zandi

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Deep Reinforcement Learning (DRL) has started showing success in real-world applications such as building energy optimization. Much of the research in this space utilized simulated environments to train RL-agent in an offline mode. Very few research have used DRL-based control in real-world systems due to two main reasons: 1) sample efficiency challenge-DRL approaches need to perform a lot of interactions with the environment to collect sufficient experiences to learn from, which is difficult in real systems, and 2) comfort or safety related constraints-user's comfort must never or at least rarely be violated. In this work, we propose a novel deep Reinforcement Learning framework with online Data Augmentation (RLDA) to address the sample efficiency challenge of real-world RL. We used a time series Generative Adversarial Network (TimeGAN) architecture as a data generator. We further evaluated the proposed RLDA framework using a case study of an intelligent HVAC control. With a ≈28% improvement in the sample efficiency, RLDA framework lays the way towards increased adoption of DRL-based intelligent control in real-world building energy management systems.

Original languageEnglish
Title of host publicationBuildSys 2022 - Proceedings of the 2022 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation
PublisherAssociation for Computing Machinery, Inc
Pages479-483
Number of pages5
ISBN (Electronic)9781450398909
DOIs
StatePublished - Nov 9 2022
Event9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys 2022 - Boston, United States
Duration: Nov 9 2022Nov 10 2022

Publication series

NameBuildSys 2022 - Proceedings of the 2022 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation

Conference

Conference9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys 2022
Country/TerritoryUnited States
CityBoston
Period11/9/2211/10/22

Funding

This research is sponsored by the Artificial Intelligence Initiative as part of the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the US Department of Energy under contract DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). RLEM ’22, November 9–10, 2022, Boston, MA, USA © 2022 Association for Computing Machinery. ACM ISBN 978-1-4503-9890-9/22/11...$15.00 https://doi.org/10.1145/3563357.3566168

Keywords

  • building energy
  • data augmentation
  • deep reinforcement learning
  • demand response
  • intelligent HVAC control

Fingerprint

Dive into the research topics of 'Deep reinforcement learning with online data augmentation to improve sample efficiency for intelligent HVAC control'. Together they form a unique fingerprint.

Cite this