The repository is for Safe Reinforcement Learning (RL) in power system applications, based on the paper A Review of Safe Reinforcement Learning Methods for Modern Power Systems, where we explore various safe RL baselines, benchmarks, applications, and real-world deployments.
If any papers are missing from this list, or if any authors prefer not to have their work included here, please feel free to contact tong.su.th@dartmouth.edu. (This repository is under active development. We welcome and appreciate any constructive comments and suggestions.)
The README is organized as follows:
- 1. RL/DRL/Safe RL Base
- 2. Benchmark Environments, Algorithms, and Software
- 3 Power System Applications of Safe RL
- 4 Real-World Deployment Cases
- Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications
- Deep reinforcement learning: An overview
- Deep reinforcement learning: A brief survey
- A brief survey of deep reinforcement learning
- Deep reinforcement learning: An overview
- Explainable AI and reinforcement learning—a systematic review of current approaches and trends
- Robust reinforcement learning: A review of foundations and recent advances
- A survey on offline reinforcement learning: Taxonomy, review, and open problems
- Deep reinforcement learning for smart grid operations: Algorithms, applications, and prospects
- Reinforcement learning for selective key applications in power systems: Recent advances and future challenges
- Deep reinforcement learning for power system applications: An overview
- (Deep) reinforcement learning for electric power system control and related problems: A short review and perspectives
- Reinforcement learning and its applications in modern power and energy systems: A review
- A review of safe reinforcement learning: Methods, theories and applications
- A comprehensive survey on safe reinforcement learning
- State-wise safe reinforcement learning: A survey
- 安全强化学习综述 (Safe reinforcement learning: A survey) PDF
- AI Safety Gridworlds
- Safety-Gym
- Safety-Gymnasium
- Safe Multi-Agent Mujoco
- Safe Multi-Agent Isaac Gym
- Safe Multi-Agent Robosuite
- Safe-Policy-Optimization(SafePO)
- OmniSafe
- PSSE: Website, Python interface for state estimation, Python interface for dynamic simulation 1, Python interface for dynamic simulation 2,
- PowerFactory: Website, Python interface
- PowerWorld: Website, WebHelp, Python interface-ESA
- EMTP: Website, Manual
- ETAP: Website
- RTDS: Website
- Simscape: Website
- PSCAD: Website, Course-Python-PSCAD
- OpenDSS: An open source platform for collaborating on smart grid research
- GridLAB-D: GridLAB-D: an agent-based simulation framework for smart grids
- MATPOWER: MATPOWER: Steady-state operations, planning, and analysis tools for power systems research and education
- Pandapower: Pandapower—an open-source python tool for convenient modeling, analysis, and optimization of electric power systems
- PyPSA: PyPSA: Python for power system analysis
- PowerModels: Powermodels.jl: An open-source framework for exploring power flow formulations
- PST: A toolbox for power system dynamics and control engineering education and research, Software and manual
- PSAT: An open source power system analysis toolbox, Software and manual
- PowerSimulations.jl: PowerSystems.jl—A power system data management package for large scale modeling
- PowerModelsDistribution.jl: PowerModelsDistribution.jl: An open-source framework for exploring distribution power flow formulations
- ANDES: Hybrid symbolic-numeric framework for power system modeling and analysis
- PowerSimulationsDynamics.jl: PowerSimulationsDynamics.jl-An open source modeling package for modern power systems with inverter-based resources
- Dynaωo: Towards an open-source solution using Modelica for time-domain simulation of power systems
- OMG: OMG: A scalable and flexible simulation and testing environment toolbox for intelligent microgrid control
- RLGC: RLGC, Adaptive power system emergency control using deep reinforcement learning
- PowerGym: PowerGym: A reinforcement learning environment for Volt-VAR control in power distribution systems
- OPF-Gym: OPF-Gym, Learning the optimal power flow: Environment design matters
- CommonPower: CommonPower, CommonPower: Supercharging machine learning for smart grids
- Data driven decentralized control of inverter based renewable energy sources using safe guaranteed multi-agent deep reinforcement learning
- Multi-agent safe graph reinforcement learning for PV inverters-based real-time decentralized volt/var control in zoned distribution networks
- Two-stage deep reinforcement learning for inverter-based Volt-VAR control in active distribution networks
- Safe off-policy deep reinforcement learning algorithm for Volt-VAR control in power distribution systems
- Volt-VAR control in power distribution systems with deep reinforcement learning
- Decentralized safe reinforcement learning for inverter-based voltage control
- Stability constrained reinforcement learning for real-time voltage control
- Physics-shielded multi-agent deep reinforcement learning for safe active voltage control with photovoltaic/battery energy storage systems
- Model-augmented safe reinforcement learning for Volt-VAR control in power distribution networks
- Online multi-agent reinforcement learning for decentralized inverter-based Volt-VAR control
- SAVER: Safe learning-based controller for real-time voltage regulation
- DNN assisted projection based deep reinforcement learning for safe control of distribution grids
- Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks
- Safe multi-agent deep reinforcement learning for real-time decentralized control of inverter based renewable energy resources considering communication delay
- Three-stage inverter-based peak shaving and Volt-VAR control in active distribution networks using online safe deep reinforcement learning
- Safety deep reinforcement learning approach to voltage control in flexible network topologies
- Explicit reinforcement learning safety layer for computationally efficient inverter-based voltage regulation
- Multi-agent primal dual DDPG based reactive power optimization of active distribution networks via graph reinforcement learning
- Human-in-the-loop reinforcement learning method for Volt/Var control in active distribution network with safe operation mechanism
- Reinforcement learning for optimal primary frequency control: A Lyapunov approach
- Online preventive control for transmission overload relief using safe reinforcement learning with enhanced spatial-temporal awareness
- Barrier function-based safe reinforcement learning for emergency control of power systems
- Bridging transient and steady-state performance in voltage control: A reinforcement learning approach with safe gradient flow
- Deep reinforcement learning-based active network management and emergency load-shedding control for power systems
- A safe policy learning-based method for decentralized and economic frequency control in isolated networked-microgrid systems
- AdapSafe: Adaptive and safe-certified deep reinforcement learning-based frequency control for carbon-neutral power systems
- Computationally efficient safe reinforcement learning for power systems
- Coordinated frequency control through safe reinforcement learning
- Coordinated wide-area damping control using deep neural networks and reinforcement learning
- Risk-constrained reinforcement learning for inverter-dominated power system controls
- Safe reinforcement learning for mitigation of model errors in facts setpoint control
- Stability-certified reinforcement learning: A control-theoretic perspective
- Recurrent neural network controllers synthesis with stability guarantees for partially observed systems
- A barrier-certificated reinforcement learning approach for enhancing power system transient stability
- Reinforcement learning for stability-guaranteed adaptive optimal primary frequency control of power systems using partially monotonic neural networks
- A Benders-combined safe reinforcement learning framework for risk-averse dispatch considering frequency security constraints
- Safe reinforcement learning-based transient stability control for islanded microgrids with topology reconfiguration
- Knowledge-GPT guided generalizable reinforcement learning for intelligent emergency generator tripping in power system
- Learning to operate distribution networks with safe deep reinforcement learning
- Multi-agent safe policy learning for power management of networked microgrids
- Safe deep reinforcement learning for microgrid energy management in distribution networks with leveraged spatial–temporal perception
- Model-free economic dispatch for virtual power plants: An adversarial safe reinforcement learning approach
- A hybrid data-driven method for fast solution of security-constrained optimal power flow
- Robust energy management system with safe reinforcement learning using short-horizon forecasts
- Constrained reinforcement learning for predictive control in real-time stochastic dynamic optimal power flow
- Online operational decision-making for integrated electric-gas systems with safe reinforcement learning
- Real-time sequential security-constrained optimal power flow: A hybrid knowledge-data-driven reinforcement learning approach
- Online microgrid energy management based on safe deep reinforcement learning
- Distributed economic dispatch in microgrids based on cooperative reinforcement learning
- Lyapunov-based safe reinforcement learning for microgrid energy management
- Real-time optimal power flow method via safe deep reinforcement learning based on primal-dual and prior knowledge guidance
- Feasibility constrained online calculation for real-time optimal power flow: A convex constrained deep reinforcement learning approach
- Improved proximal policy optimization algorithm for sequential security-constrained optimal power flow based on expert knowledge and safety layer
- Networked multiagent-based safe reinforcement learning for low-carbon demand management in distribution networks
- Optimal energy system scheduling using a constraint-aware reinforcement learning algorithm
- Real-time optimal power flow with linguistic stipulations: Integrating GPT-agent and deep reinforcement learning
- Real-time optimal power flow: A Lagrangian based deep reinforcement learning approach
- Safe reinforcement learning for multi-energy management systems with known constraint functions
- Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach
- Safe multi-agent deep reinforcement learning for decentralized low-carbon operation in active distribution networks and multi-microgrids
- Active sensitivity coefficient-guided reinforcement learning for power grid real-time dispatching
- Privacy-enhanced safe reinforcement learning for the dispatch of a local energy community
- Carbon cap based multi-energy sharing among heterogeneous microgrids using multi-agent safe reinforcement learning method with credit assignment and sequential update
- RL2: Reinforce large language model to assist safe reinforcement learning for energy management of active distribution networks
- Active sensitivity coefficient-guided reinforcement learning for power grid real-time dispatching
- Multi-agent safe reinforcement learning based distributed optimal dispatch for active distribution network with incomplete information
- Deep reinforcement learning approach for dynamic distribution network reconfiguration based on sequential masking
- Knowledge-augmented population-based deep reinforcement learning for real-time network-constrained economic dispatch of large-scale power grid
- A robust safe reinforcement learning-based operation method for hybrid electric-hydrogen energy system risk-based dispatch considering dynamic efficiency characteristics of electrolysers
- Coordinated operation of multi-energy microgrids considering green hydrogen and congestion management via a safe policy learning approach
- Physics-shielded deep reinforcement learning for safe energy management of microgrids with battery health consideration
- Distributed robust dispatch for networked microgrids with coalition game-guided multiagent adversarial safe reinforcement learning
- A safe combined reinforcement learning and model predictive control scheme for utility-level battery control in distribution grids
- Shield-enhanced safe reinforcement learning control for wave energy converters
- Deep reinforcement learning from demonstrations to assist service restoration in islanded microgrids
- Primal-dual differentiable programming for distribution system critical load restoration
- Safe exploration reinforcement learning for load restoration using invalid action masking
- Safe deep reinforcement learning for resilient self-proactive distribution grids against wildfires
- Deep reinforcement learning based unit commitment scheduling under load and wind power uncertainty
- Risk-based reserve scheduling for active distribution networks based on an improved proximal policy optimization algorithm
- Soft actor-critic combined with logic-based Benders decomposition algorithm for monthly security constrained unit commitment under wind power uncertainty
- An augmented Lagrangian-based safe reinforcement learning algorithm for carbon-oriented optimal scheduling of EV aggregators
- A budget-aware incentive mechanism for vehicle-to-grid via reinforcement learning
- Dynamic incentive pricing on charging stations for real-time congestion management in distribution network: an adaptive model-based safe deep reinforcement learning method
- SMA-PDPPO: Safe multiagent primal-dual deep reinforcement learning for industrial parks energy trading
- Network-constrained P2P trading: A safety-aware decentralized multi-agent reinforcement learning approach
- Network-constrained reinforcement learning for optimal EV charging control
- Data-driven coordinated charging for electric vehicles with continuous charging rates: A deep policy gradient approach
- A deep reinforcement learning-based charging scheduling approach with augmented Lagrangian for electric vehicles
- A safe reinforcement learning-based charging strategy for electric vehicles in residential microgrid
- Constrained EV charging scheduling based on safe deep reinforcement learning
- A deep reinforcement learning-based energy management framework with Lagrangian relaxation for plug-in hybrid electric vehicle
- Safe deep reinforcement learning hybrid electric vehicle energy management
- Rule-based shields embedded safe reinforcement learning approach for electric vehicle charging control
- Safe-AutoSAC: AutoML-enhanced safe deep reinforcement learning for integrated energy system scheduling with multi-channel informer forecasting and electric vehicle demand response
- Model-free safe deep reinforcement learning for grid-to-vehicle management considering grid constraints and transformer thermal stress
- Reinforcement learning with dual safety policies for energy savings in building energy systems
- Residual physics and post-posed shielding for safe deep reinforcement learning method
- Optimal dispatch of an energy hub with compressed air energy storage: A safe reinforcement learning approach
- A safe reinforcement learning approach for multi-energy management of smart home
- Deep reinforcement learning for tropical air free-cooled data center control
- DRL-S: Toward safe real-world learning of dynamic thermal management in data center
- District cooling system control for providing operating reserve based on safe deep reinforcement learning
- Safe building HVAC control via batch reinforcement learning
- Safe reinforcement learning-based resilient proactive scheduling for a commercial building considering correlated demand response
- Safe reinforcement learning for real-time automatic control in a smart energy-hub
- Energy management based on safe multi-agent reinforcement learning for smart buildings in distribution networks
- Safe deep reinforcement learning for building energy management
- Green data center cooling control via physics-guided safe reinforcement learning
- SafeCool: Safe and energy-efficient cooling management in data centers with model-based reinforcement learning
- Toward model-assisted safe reinforcement learning for data center cooling control: A Lyapunov-based approach
- GT Auto Tuner: GT Auto Tuner
- Building Cooling Systems: Google, DeepMind, Controlling commercial cooling systems using reinforcement learning, TELUS and Vector Institute, SAB
- DeepThermal: Deepthermal: Combustion optimization for thermal power generating units using offline reinforcement learning
If you find the repository useful, please cite the paper:
@article{su2025review,
title={A review of safe reinforcement learning methods for modern power systems},
author={Su, Tong and Wu, Tong and Zhao, Junbo and Scaglione, Anna and Xie, Le},
journal={Proceedings of the IEEE},
year={2025},
volume={113},
number={3},
pages={213-255},
doi={10.1109/JPROC.2025.3584656}
}
@article{su2024review,
title={A review of safe reinforcement learning methods for modern power systems},
author={Su, Tong and Wu, Tong and Zhao, Junbo and Scaglione, Anna and Xie, Le},
journal={arXiv preprint arXiv:2407.00304},
year={2024}
}