Reinforcement learning based energy efficient robot relay for unmanned aerial vehicles against smart jamming

作     者：Xiaozhen LU Jingfang JIE Zihan LIN Liang XIAO Jin LI Yanyong ZHANG Xiaozhen LU;Jingfang JIE;Zihan LIN;Liang XIAO;Jin LI;Yanyong ZHANG

作者机构：Department of Information and Communication Engineering Xiamen University School of Computer Science and Technology University of Science and Technology of China School of Computer Science and Educational Software Guangzhou University 

出 版 物：《Science China(Information Sciences)》 (中国科学:信息科学(英文版))

年 卷 期：2022年第65卷第1期

页      面：234-246页

核心收录：

学科分类：080202[工学-机械电子工程] 08[工学] 0804[工学-仪器科学与技术] 082503[工学-航空宇航制造工程] 0802[工学-机械工程] 0825[工学-航空宇航科学与技术] 

基　　金：supported by National Natural Science Foundation of China (Grant Nos. 61971366, 61731012) Fundamental Research Funds for the Central Universities (Grant No. 20720200077) Natural Science Foundation of Fujian Province of China (Grant No. 2020J01430) 

主　　题：unmanned aerial vehicles relay jamming game theory reinforcement learning 

摘      要：Unmanned aerial vehicles(UAVs) with limited energy resources, severe path loss, and shadowing to the ground base stations are vulnerable to smart jammers that aim to degrade the UAV communication performance and exhaust the UAV energy. The UAV anti-jamming communication performance, such as the outage probability, degrades if the robot relay is not aware of the jamming policies and the UAV network topology. In this paper, we propose a robot relay scheme for UAVs against smart jamming, which combines reinforcement learning with a function approximation approach named tile coding, to jointly optimize the robot moving distance and relay power with the unknown jamming channel states and locations. The robot mobility and relay policy are chosen based on the received jamming power, the robot received signal quality,location and energy consumption, and the bit error rate of the UAV messages. We also present a deep reinforcement learning version for the robot with sufficient computing resources. It uses three deep neural networks to choose the robot mobility and relay policy with reduced sample complexity, so as to avoid exploring dangerous policies that lead to the high outage probability of the UAV messages. The network architecture of the three networks is designed with fully connected layers instead of convolutional layers to reduce the computational complexity, which is analyzed by theoretical analyses. We provide the performance bound of the proposed schemes in terms of the bit error rate, robot energy consumption and utility based on a game-theoretic study. Simulation results show that the performance of our proposed relay schemes,including the bit error rate, the outage probability, and the robot energy consumption outperforms the existing schemes.