Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

Alaa Selim; Junbo Zhao; Jin Dong; Jianming Lian

doi:10.1109/TIA.2025.3626472

Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

Alaa Selim
, Junbo Zhao
, Jin Dong
, Jianming Lian

Grid-Interactive Controls

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

This paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.

Original language	English
Pages (from-to)	3635-3647
Number of pages	13
Journal	IEEE Transactions on Industry Applications
Volume	62
Issue number	2
DOIs	https://doi.org/10.1109/TIA.2025.3626472
State	Published - 2026

Funding

This work was supported in part by the US Department of Energy Office of Electricity, Advanced Grid Modeling (AGM) Program, and in part by the Solar Energy Technologies Office. This manuscript has been authored in part by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the work for publication, acknowledges that the US government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the submitted manuscript version of this work, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan. This manuscript has been authored in part by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the work for publication, acknowledges that the US government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the submitted manuscript version of this work, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan.

Keywords

Safe deep reinforcement learning
black-start
constrained markov decision process (CMDP)
microgrid restoration
soft actor-critic (SAC)

Access to Document

10.1109/TIA.2025.3626472

Cite this

@article{ea7258105694429fb2569ae2c41538fe,

title = "Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration",

abstract = "This paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5\% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.",

keywords = "Safe deep reinforcement learning, black-start, constrained markov decision process (CMDP), microgrid restoration, soft actor-critic (SAC)",

author = "Alaa Selim and Junbo Zhao and Jin Dong and Jianming Lian",

note = "Publisher Copyright: {\textcopyright} 1972-2012 IEEE.",

year = "2026",

doi = "10.1109/TIA.2025.3626472",

language = "English",

volume = "62",

pages = "3635--3647",

journal = "IEEE Transactions on Industry Applications",

issn = "0093-9994",

publisher = "Institute of Electrical and Electronics Engineers",

number = "2",

}

TY - JOUR

T1 - Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

AU - Selim, Alaa

AU - Zhao, Junbo

AU - Dong, Jin

AU - Lian, Jianming

PY - 2026

Y1 - 2026

N2 - This paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.

AB - This paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.

KW - Safe deep reinforcement learning

KW - black-start

KW - constrained markov decision process (CMDP)

KW - microgrid restoration

KW - soft actor-critic (SAC)

UR - https://www.scopus.com/pages/publications/105020422634

U2 - 10.1109/TIA.2025.3626472

DO - 10.1109/TIA.2025.3626472

M3 - Article

AN - SCOPUS:105020422634

SN - 0093-9994

VL - 62

SP - 3635

EP - 3647

JO - IEEE Transactions on Industry Applications

JF - IEEE Transactions on Industry Applications

IS - 2

ER -

Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this