Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning

作     者：Mingcheng Zhang Chao Yan Wei Dai Xiaojia Xiang Kin Huat Low 

作者机构：School of Mechanical and Aerospace EngineeringNanyang Technological UniversitySingapore 639798Singapore College of Intelligence Science and TechnologyNational University of Defense TechnologyChangsha 410073China Air Traffic Management Research InstituteNanyang Technological UniversitySingapore 637460Singapore 

出 版 物：《Green Energy and Intelligent Transportation》 (新能源与智能载运（英文）)

年 卷 期：2023年第2卷第4期

页      面：43-57页

核心收录：

学科分类：08[工学] 0823[工学-交通运输工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Research Foundation(NRF) Singapore and the Civil Aviation Authority of Singapore(CAAS) under the Aviation Transformation Programme(ATP) 

主　　题：Unmanned aircraft system traffic management Tactical conflict resolution Double deep Q network Attention mechanism Secondary conflict 

摘      要：Unmanned aerial vehicles(UAVs)have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace.A traffic management system for these UAVs is needed to manage this future *** conflict resolution for unmanned aerial systems(UASs)is an essential piece of the puzzle for the future UAS Traffic Management(UTM),especially in very low-level(VLL)urban *** conflict resolution in higher altitude airspace,the dense high-rise buildings are an essential source of potential conflict to be considered in VLL urban *** this paper,we propose an attention-based deep reinforcement learning approach to solve the tactical conflict resolution ***,we formulate this task as a sequential decision-making problem using Markov Decision Process(MDP).The double deep Q network(DDQN)framework is used as a learning framework for the host drone to learn to output conflict-free maneuvers at each time *** use the attention mechanism to model the individual neighbor s effect on the host drone,endowing the learned conflict resolution policy to be adapted to an arbitrary number of neighboring ***,we build a simulation environment with various scenarios covering different types of encounters to evaluate the proposed *** simulation results demonstrate that our proposed algorithm provides a reliable solution to minimize secondary conflict counts compared to learning and non-learning-based approaches under different traffic density scenarios.