Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

作     者：Lei Dong Jing Wei Hao Lin Xinying Wang Lei Dong;Jing Wei;Hao Lin;Xinying Wang

作者机构：School of Electric EngineeringNorth China Electric Power UniversityChangping DistrictBeijing 102206P.R.China China Electric Power Research InstituteHaidian DistrictBeijing 100192P.R.China 

出 版 物：《Global Energy Interconnection》 (全球能源互联网（英文版）)

年 卷 期：2022年第5卷第6期

页      面：604-617页

核心收录：

学科分类：080702[工学-热能工程] 0808[工学-电气工程] 080802[工学-电力系统及其自动化] 08[工学] 081104[工学-模式识别与智能系统] 0807[工学-动力工程及工程热物理] 0811[工学-控制科学与工程] 

基　　金：supported by The National Key R&D Program of China(2020YFB0905900):Research on artificial intelligence application of power internet of things. 

主　　题：Integrated energy system Multi-agent system Distributed optimization Multi-agent deep deterministic policy gradient Real-time optimization decision 

摘      要：The coordinated optimization problem of the electricity-gas-heat integrated energy system(IES)has the characteristics of strong coupling,non-convexity,and nonlinearity.The centralized optimization method has a high cost of communication and complex modeling.Meanwhile,the traditional numerical iterative solution cannot deal with uncertainty and solution efficiency,which is difficult to apply online.For the coordinated optimization problem of the electricity-gas-heat IES in this study,we constructed a model for the distributed IES with a dynamic distribution factor and transformed the centralized optimization problem into a distributed optimization problem in the multi-agent reinforcement learning environment using multi-agent deep deterministic policy gradient.Introducing the dynamic distribution factor allows the system to consider the impact of changes in real-time supply and demand on system optimization,dynamically coordinating different energy sources for complementary utilization and effectively improving the system economy.Compared with centralized optimization,the distributed model with multiple decision centers can achieve similar results while easing the pressure on system communication.The proposed method considers the dual uncertainty of renewable energy and load in the training.Compared with the traditional iterative solution method,it can better cope with uncertainty and realize real-time decision making of the system,which is conducive to the online application.Finally,we verify the effectiveness of the proposed method using an example of an IES coupled with three energy hub agents.