On UAV Serving Node Deployment for Temporary Coverage in Forest Environment:A Hierarchical Deep Reinforcement Learning Approach

作     者：WANG Li WU Xuewei WANG Yanhui XIAO Zhe LI Liang FEI Aiguo WANG Li;WU Xuewei;WANG Yanhui;XIAO Zhe;LI Liang;FEI Aiguo

作者机构：School of Computer Science (National Pilot Software Engineering School) Beijing University of Posts and Telecommunications School of Electronic Engineering Beijing University of Posts and Telecommunications 

出 版 物：《Chinese Journal of Electronics》 (电子学报(英文))

年 卷 期：2023年第32卷第4期

页      面：760-772页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 0907[农学-林学] 081104[工学-模式识别与智能系统] 08[工学] 0829[工学-林业工程] 09[农学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the Beijing Municipal Natural Science Foundation (L192030) the National Key Research and Development Program of China (2020YFC1511801) the National Natural Science Foundation of China (62171054, U2066201, 61871416) 

主　　题：Deep learning Base stations Simulation Clustering algorithms Forestry Reinforcement learning Autonomous aerial vehicles 

摘      要：Unmanned aerial vehicles(UAVs) can be effectively used as serving stations in emergency communications because of their free movements, strong flexibility, and dynamic coverage. In this paper, we propose a coordinated multiple points based UAV deployment framework to improve system average ergodic rate, by using the fuzzy C-means algorithm to cluster the ground users and considering exclusive forest channel models for the two cases, i.e., associated with a broken base station or an available base station. In addition, we derive the upper bound of the average ergodic rate to reduce computational complexity. Since deep reinforcement learning(DRL) can deal with the complex forest environment while the large action and state space of UAVs leads to slow convergence, we use a ratio cut method to divide UAVs into groups and propose a hierarchical clustering DRL(HC-DRL) approach with quick convergence to optimize the UAV deployment. Simulation results show that the proposed framework can effectively reduce the complexity, and outperforms the counterparts in accelerating the convergence speed.