Abstract
The future of grid control requires a hybrid approach combining centralized and decentralized methods to fully utilize the potential of smart edge devices with artificial intelligence (AI) capabilities. This paper aims to develop and evaluate a federated deep reinforcement learning (FDRL) framework for decentralized adaptive volt-var optimization (VVO) of behind-the-meter (BTM) distributed energy resources (DERs). First, this paper models a single deep reinforcement learning (DRL) agent using the Markov Decision Process (MDP) framework for decentralized adaptive VVO of BTM DERs. Two DRL algorithms, soft actor-critic (SAC) and twin-delayed deep deterministic policy gradient (TD3), are compared for their effectiveness in optimizing VVO. Results show that TD3 outperforms SAC, achieving a 71.3% improvement in mean reward. Finally, the DRL agent is deployed within the FDRL framework, using the Flower platform, to enhance learning, provide adaptive control, and ensure data privacy for BTM DERs.
| Original language | English |
|---|---|
| Title of host publication | 2024 56th North American Power Symposium, NAPS 2024 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| ISBN (Electronic) | 9798331521035 |
| DOIs | |
| State | Published - 2024 |
| Event | 56th North American Power Symposium, NAPS 2024 - El Paso, United States Duration: Oct 13 2024 → Oct 15 2024 |
Publication series
| Name | 2024 56th North American Power Symposium, NAPS 2024 |
|---|
Conference
| Conference | 56th North American Power Symposium, NAPS 2024 |
|---|---|
| Country/Territory | United States |
| City | El Paso |
| Period | 10/13/24 → 10/15/24 |
Funding
This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doepublic- access-plan). This work has been supported in part by DOE s Office of Electricity.
Keywords
- behind-the-meter
- deep reinforcement learning
- distributed resources
- federated learning
- soft actor-critic
- twin-delayed deep deterministic policy gradient
- voltage control