Scalable Load Management Using Reinforcement Learning

The project objective is to develop a cost-effective load management system that is easy to deploy on existing homes, can enable interoperability, and optimize the loads to meet the homeowners' comforts and economic constraints. This project will develop and utilize a distributed agent-based reinforcement learning (RL) algorithm to understand the energy usage by building occupants and generate the optimized schedule for different loads within acceptable time-interval to satisfy the homeowner comfort constraints, minimize the energy cost, and ensure a reliable service for the grid.

In this algorithm, the house agent learns how to choose its next action to maximize its future benefit and rewards. The effort includes defining and formulating the parameters for the learning algorithm including the state space, the action space, the transition functions, and the reward function using a Markov Decision Process (MDP). The scalability, performance, and accuracy of the algorithm will first be evaluated using the simulation environment that has been developed at ORNL for modeling the integrated control strategies that appear in smart grid applications. In this simulator, the realistic thermal behavior of each home and device is simulated using the RC model that has been trained using EnergyPlus data. The load management system will be deployed at ORNL Yarnell Station research house to demonstrate its performance and energy-saving capabilities.

Project Impact

The software and control algorithms developed as part of this research will provide a software framework that can be utilized in existing homes to enable load flexibility for reducing cost, improving energy efficiency, improving occupant comfort, and improving grid resiliency with minimal effort and minimal additional devices. This project provides data analysis that evaluates the benefit for RL-based control strategies for existing homes.