UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

作     者：ZHANG Jiandong YANG Qiming SHI Guoqing LU Yi WU Yong ZHANG Jiandong;YANG Qiming;SHI Guoqing;LU Yi;WU Yong

作者机构：School of Electronics and InformationNorthwestern Polytechnical UniversityXi’an 710072China Shenyang Aircraft Design InstituteShenyang 110035China 

出 版 物：《Journal of Systems Engineering and Electronics》 (系统工程与电子技术（英文版）)

年 卷 期：2021年第32卷第6期

页      面：1421-1438页

核心收录：

学科分类：11[军事学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 0808[工学-电气工程] 08[工学] 081104[工学-模式识别与智能系统] 082503[工学-航空宇航制造工程] 0825[工学-航空宇航科学与技术] 0811[工学-控制科学与工程] 1109[军事学-军事装备学] 

基　　金：supported by the Aeronautical Science Foundation of China(2017ZC53033) the Seed Foundation of Innovation and Creation for Graduate Students in Northwestern Polytechnical University(CX2020156)。 

主　　题：decision-making air combat maneuver cooperative air combat reinforcement learning recurrent neural network 

摘      要：In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.