MADRL-based UAV swarm non-cooperative game under incomplete information

作     者：Ershen WANG Fan LIU Chen HONG Jing GUO Lin ZHAO Jian XUE Ning HE Ershen WANG;Fan LIU;Chen HONG;Jing GUO;Lin ZHAO;Jian XUE;Ning HE

作者机构：School of Electronic and Information EngineeringShenyang Aerospace UniversityShenyang 110136China College of RoboticsBeijing Union UniversityBeijing 100101China School of Engineering ScienceUniversity of Chinese Academy of SciencesBeijing 100049China College of Smart CityBeijing Union UniversityBeijing 100101China 

出 版 物：《Chinese Journal of Aeronautics》 (中国航空学报（英文版）)

年 卷 期：2024年第37卷第6期

页      面：293-306页

核心收录：

学科分类：12[管理学] 08[工学] 0710[理学-生物学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 082503[工学-航空宇航制造工程] 0835[工学-软件工程] 0714[理学-统计学（可授理学、经济学学位）] 0825[工学-航空宇航科学与技术] 0836[工学-生物工程] 0701[理学-数学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 081202[工学-计算机软件与理论] 

基　　金：supported by the National Key R&D Program of China(No.2018AAA0100804) the National Natural Science Foundation of China(No.62173237) the Academic Research Projects of Beijing Union University,China(Nos.SK160202103,ZK50201911,ZK30202107,ZK30202108) the Song Shan Laboratory Foundation,China(No.YYJC062022017) the Applied Basic Research Programs of Liaoning Province,China(Nos.2022020502-JH2/1013,2022JH2/101300150) the Special Funds program of Civil Aircraft,China(No.01020220627066) the Special Funds program of Shenyang Science and Technology,China(No.22-322-3-34) 

主　　题：UAV swarm Reinforcement learning Deep learning Multi-agent Non-cooperative game Nash equilibrium 

摘      要：Unmanned Aerial Vehicles(UAVs)play increasing important role in modern *** this paper,considering the incomplete observation information of individual UAV in complex combat environment,we put forward an UAV swarm non-cooperative game model based on Multi-Agent Deep Reinforcement Learning(MADRL),where the state space and action space are constructed to adapt the real features of UAV swarm air-to-air *** multi-agent particle environment is employed to generate an UAV combat scene with continuous observation *** recently popular MADRL methods are compared extensively in the UAV swarm noncooperative game model,the results indicate that the performance of Multi-Agent Soft Actor-Critic(MASAC)is better than that of other MADRL methods by a large *** swarm employing MASAC can learn more effective policies,and obtain much higher hit rate and win *** under different swarm sizes and UAV physical parameters are also performed,which implies that MASAC owns a well generalization ***,the practicability and convergence of MASAC are addressed by investigating the loss value of Q-value networks with respect to individual UAV,the results demonstrate that MASAC is of good practicability and the Nash equilibrium of the UAV swarm non-cooperative game under incomplete information can be reached.