检索结果-南通市图书馆

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

Science China(Information Sciences) 2024年第3期67卷 225-245页

作者： Xuanhan ZHOU Jun XIONG Haitao ZHAO Xiaoran LIU Baoquan REN Xiaochen ZHANG Jibo WEI Hao YIN College of Electronic Science and Technology National University of Defense Technology Systems Engineering Institute Academy of Military Sciences PLA

Unmanned aerial vehicles(UAVs) are recognized as effective means for delivering emergency communication services when terrestrial infrastructures are unavailable. This paper investigates a multiUAV-assisted communication system, where we jointly optimize UAVs’ trajectories, user association, and ground users(GUs)’ transmit power to maximize a defined fairness-weighted throughput metric. Owing to the dynamic nature of UAVs, this problem has to be solved in real time. However, the problem’s non-convex and combinatorial attributes pose challenges for conventional optimization-based algorithms, particularly in scenarios without central controllers. To address this issue, we propose a multi-agent deep reinforcement learning(MADRL) approach to provide distributed and online solutions. In contrast to previous MADRLbased methods considering only UAV agents, we model UAVs and GUs as heterogeneous agents sharing a common objective. Specifically, UAVs are tasked with optimizing their trajectories, while GUs are responsible for selecting a UAV for association and determining a transmit power level. To learn policies for these heterogeneous agents, we design a heterogeneous coordinated QMIX(HC-QMIX) algorithm to train local Q-networks in a centralized manner. With these well-trained local Q-networks, UAVs and GUs can make individual decisions based on their local observations. Extensive simulation results demonstrate that the proposed algorithm outperforms state-of-the-art benchmarks in terms of total throughput and system fairness.

关键词： unmanned aerial vehicle (UAV) trajectory design resource allocation multi-agent deep reinforcement learning (MADRL) heterogeneous agents

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

trajectory design for UAV-Enabled Maritime Secure Communications:A Reinforcement Learning Approach

引用

China Communications 2022年第9期19卷 26-36页

作者： Jintao Liu Feng Zeng Wei Wang Zhichao Sheng Xinchen Wei Kanapathippillai Cumanan School of Information Science and Technology Nantong UniversityNantong 226019China Nantong Research Institute for Advanced Communication Technologies Nantong 226019China Key Laboratory of Specialty Fiber Optics and Optical Access Networks Shanghai UniversityShanghai 200444China Department of Electronic Engineering University of YorkYorkYO105DDUnited Kingdom

This paper investigates an unmanned aerial vehicle(UAV)-enabled maritime secure communication network,where the UAV aims to provide the communication service to a legitimate mobile vessel in the presence of multiple eavesdroppers.In this maritime communication networks(MCNs),it is challenging for the UAV to determine its trajectory on the ocean,since it cannot land or replenish energy on the sea surface,the trajectory should be pre-designed before the UAV takes off.Furthermore,the take-off location of the UAV and the sea lane of the vessel may be random,which leads to a highly dynamic environment.To address these issues,we propose two reinforcement learning schemes,Q-learning and deep deterministic policy gradient(DDPG)algorithms,to solve the discrete and continuous UAV trajectory design problem,respectively.Simulation results are provided to validate the effectiveness and superior performance of the proposed reinforcement learning schemes versus the existing schemes in the literature.Additionally,the proposed DDPG algorithm converges faster and achieves higher utilities for the UAV,compared to the Q-learning algorithm.

关键词： maritime communication networks(MCNs) unmanned aerial vehicles(UAV) reinforcement learning physical layer security trajectory design

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Multi-Agent Few-Shot Meta Reinforcement Learning for trajectory design and Channel Selection in UAV-Assisted Networks

引用

China Communications 2022年第4期19卷 166-176页

作者： Shiyang Zhou Yufan Cheng Xia Lei Huanhuan Duan National Key Laboratory of Science and Technology on Communications University of Electronic Science and Technology of ChinaChengdu 611731China

Unmanned aerial vehicle(UAV)-assisted communications have been considered as a solution of aerial networking in future wireless networks due to its low-cost, high-mobility, and swift features. This paper considers a UAV-assisted downlink transmission,where UAVs are deployed as aerial base stations to serve ground users. To maximize the average transmission rate among the ground users, this paper formulates a joint optimization problem of UAV trajectory design and channel selection, which is NP-hard and non-convex. To solve the problem, we propose a multi-agent deep Q-network(MADQN) scheme.Specifically, the agents that the UAVs act as perform actions from their observations distributively and share the same reward. To tackle the tasks where the experience is insufficient, we propose a multi-agent meta reinforcement learning algorithm to fast adapt to the new tasks. By pretraining the tasks with similar distribution, the learning model can acquire general knowledge. Simulation results have indicated the MADQN scheme can achieve higher throughput than fixed allocation. Furthermore, our proposed multiagent meta reinforcement learning algorithm learns the new tasks much faster compared with the MADQN scheme.

关键词： UAV trajectory design channel selection MADQN meta reinforcement learning

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

trajectory design and simulation in injection phase for hyper-velocity kinetic energy missile

引用

Journal of Beijing Institute of Technology 2011年第1期20卷 18-22页

作者：于剑桥梅跃松韩朝 School of Aerospace Engineering Beijing Institute of Technology

A movement law of laser beam facula is designed for the injection trajectory of hyper-ve- locity kinetic energy missile to eliminate the influence of motor exhaust smoke on laser signal trans mission. Taking guidance loop of hyper velocity kinetic energy missile as plant, a closed loop control system with desired step response characteristics is constructed and the movement law of laser beam facula for the missile injection trajectory is designed based on the output signal of the closed loop controller under a step input. Six degree of freedom trajectory simulations show that by the guidance of the laser beam facula moving with designed law, the missile can finish transition from the initial trajectory to a stable tracking trajectory without overshoot within the required time.

关键词： trajectory design trajectory simulation missile control hyper velocity kinetic energymissile

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Energy-Efficient UAV trajectory design for Backscatter Communication: A Deep Reinforcement Learning Approach

引用

China Communications 2020年第10期17卷 129-141页

作者： Yiwen Nie Junhui Zhao Jun Liu Jing Jiang Ruijin Ding School of Electronic and Information Engineering Beijing Jioatong UniversityBeijing 100044China School of Information Engineering East China Jiaotong UniversityNanchang 330013China Institute of Network Sciences and Cyberspace Tsinghua UniversityBeijing 100084China Beijing National Research Center for Information Science and Technology(BNRist) Beijing 100084China Shaanxi Key Laboratory of Information Communication Network and Security Xi’an University of Posts and TelecommunicationsXi’an 710121China Institute for Artificial Intelligence Tsinghua University(THUAI)Beijing 100084China State Key Laboratory of Intelligent Technologies and Systems Tsinghua UniversityBeijing 100084China Beijing National Research Center for Information Science and Technology(BNRist) Department of AutomationTsinghua UniversityBeijing 100084China

Recently,backscatter communication(BC)has been introduced as a green paradigm for Internet of Things(IoT).Meanwhile,unmanned aerial vehicles(UAVs)can serve as aerial base stations(BSs)to enhance the performance of BC system thanks to their high mobility and flexibility.In this paper,we investigate the problem of energy efficiency(EE)for an energy-limited backscatter communication(BC)network,where backscatter devices(BDs)on the ground harvest energy from the wireless signal of a flying rotary-wing quadrotor.Specifically,we first reformulate the EE optimization problem as a Markov decision process(MDP)and then propose a deep reinforcement learning(DRL)algorithm to design the UAV trajectory with the constraints of the BD scheduling,the power reflection coefficients,the transmission power,and the fairness among BDs.Simulation results show the proposed DRL algorithm achieves close-to-optimal performance and significant EE gains compared to the benchmark schemes.

关键词： unmanned aerial vehicle(UAV) trajectory design backscatter communication deep reinforcement learning energy-efficient

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Inverse Kinematics Analysis of a 7-DOF Space Manipulator for trajectory design

引用

Journal of Beijing Institute of Technology 2017年第3期26卷 285-291页

作者： Pengfei Xin Jili Rong Yongtai Yang Dalin Xiang School of Aerospace Engineering Beijing Institute of Technology Quanzhou Institute of Equipment Manufacturing Haixi InstitutesChinese Academy of Sciences

To solve the inverse kinematics problem for redundant degrees of freedom（DOFs）manipulators has been and still continues to be quite challenging in the field of robotics.Aiming at trajectory planning for a 7-DOF space manipulator system,joint rotation trajectories are obtained from predetermined motion trajectories and poses of the end effector in Cartesian space based on the proposed generalized inverse kinematics method.A minimum norm method is employed to choose the best trajectory among available trajectories.Numerical simulations with the7-DOF manipulator show that the proposed method can achieve the planned trajectory and pose under the circumstances of minimum angular velocities.Moreover,trajectory results from the proposed kinematics model and inverse kinematics method has the advantages of simple modelling,low computation cost,easy to solve and plan trajectory conveniently.The smooth and continuous joint rotation functions obtained from the proposed method are suitable for practical engineering applications.

关键词： redundant manipulator inverse kinematics trajectory design minimum norm method

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Modified shape-based method for three-dimensional trajectory design

引用

Journal of Harbin Institute of Technology(New Series) 2012年第2期19卷 1-4页

作者：程兴史晓宁崔乃刚 Dept.Astronautical Engineering Harbin Institute of Technology

Although the shape-based method has been proven to be useful for low-thrust trajectory design,and be capable to provide near-optimal solution for a more accurate trajectory optimization method,it is slightly non-effective when used in some 3D cases.In this paper,a modified 3D shape-based method is proposed for earth trajectory design.In this approach,in consideration of the sinusoidal periodic variation in z direction of actual trajectory,a new exponential sinusoid model is chosen for the out-of-plane motion,with four coefficients such that four scalar out-of-plane boundary conditions can be satisfied.After deriving the 3D shape-based procedure,low-thrust trajectory design example with modest inclination change is given.The results demonstrate that this modified approach is feasible for the transfer trajectory design,and comparing to the former shape-based method,the z direction solution is more coincident with the actual situation,furthermore,the solution may be used for further mission planning,trajectory evaluation and optimization.

关键词： modified shape-based three dimensional low-thrust trajectory design

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Optimized trajectory design in UAV Based Cellular Networks for 3D Users: A Double Q-Learning Approach

引用

Journal of Communications and Information Networks 2019年第1期4卷 24-32页

作者： Xuanlin Liu Mingzhe Chen Changchuan Yin Beijing University of Posts and Telecommunications Beijing 100876China

In this paper,the problem of trajectory de-sign of unmanned aerial vehicles(UAVs)for maximizing the number of satisfied users is studied in a UAV based cellular network where the UAV works as a flying base station that serves users,and the user indicates its satis-faction in terms of completion of its data request within an allowable maximum waiting time.The trajectory design is formulated as an optimization problem whose goal is to maximize the number of satisfied users.To solve this problem,a machine learning framework based on double Q-learning algorithm is proposed.The algorithm enables the UAV tofind the optimal trajectory that maximizes the number of satisfied users.Compared to the traditional learning algorithms,such as Q-learning that selects and evaluates the action using the same Q-table,the proposed algorithm can decouple the selection from the evaluation,therefore avoid overestimation which leads to sub-optimal policies.Simulation results show that the proposed algorithm can achieve up to 19.4% and 14.1% gains in terms of the number of satisfied users compared to random algorithm and Q-learning algorithm.

关键词： UAV communication trajectory design double Q-learning algorithm user satisfaction cellular network

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

trajectory design for a solar-sail mission to asteroid 2016 HO_(3)

引用

Astrodynamics 2019年第3期3卷 231-246页

作者： Jeannette Heiligers Juan M.Fernandez Olive R.Stohlman W.Keats Wilkie Faculty of Aerospace Engineering Delft University of TechnologyKluyverweg 12629 HSDelftthe Netherlands Structural Dynamics Branch Langley Research CenterNational Aeronautics and Space AdministrationHamptonVirginia23681-2199USA

This paper proposes the use of solar-sail technology currently under development at NASA Langley Research Center for a CubeSat rendezvous mission with asteroid 2016 HO_(3),a quasi-satellite of Earth.Time-optimal trajectories are sought for within a 2022–202_(3) launch window,starting from an assumed launcher ejection condition in the Earth-Moon system.The optimal control problem is solved through a particular implementation of a direct pseudo-spectral method for which initial guesses are generated through a relatively simple and straightforward genetic algorithm search on the optimal launch date and sail attitude.The results show that the trajectories take 2.16–4.21 years to complete,depending on the assumed solar-sail reflectance model and solar-sail technology.To assess the performance of solar-sail propulsion for this mission,the trajectory is also designed assuming the use of solar electric propulsion.The resulting fuel-optimal trajectories take longer to complete than the solar-sail trajectories and require a propellant consumption that exceeds the expected propellant capacity onboard the CubeSat.This comparison demonstrates the superior performance of solar-sail technology for this mission.

关键词： asteroid 2016 HO_(3) solar sail solar electric propulsion trajectory design trajectory optimization

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Effect of variable-angle trajectory structure on mechanical performance of CF/PEEK laminates made by robotic fiber placement

引用

Chinese Journal of Aeronautics 2022年第12期35卷 336-356页

作者： Shouzheng SUN Haiyang MEI Yang WANG Jiahai ZHANG Zhenyu HAN Jinglan GUO State Key Laboratory of Robotics and System Harbin Institute of TechnologyHarbin 150001China Wuhu Robot Technology Research Institute Harbin Institute of TechnologyWuhu 241000China Shanghai Spaceflight Precision Machinery Institute Shanghai 201600China School of Mechanical and Power Engineering Harbin University of Science and TechnologyHarbin 150080China

In comparison to the traditional fixed-angle trajectory,the variable-angle trajectory has a greater design space.However,it is a challenge to determine which common design curve structure is the most effective for improving mechanical performance.This work explores the effects of various design curves such as fixed-angle curve,linear curve,arc curve,sine curve,Bezier curve,and cubic polynomial curve trajectories on mechanical performance of laminates,including vibration modal performance and buckling performance.Genetic algorithm and improved NSGA-II algorithm are then used to optimize various curve structures.The results are confirmed utilizing thermoplastic Carbon Fiber(CF)/Polyether-Ether-Ketone(PEEK)laminates made by robotic fiber placement experiments.The relationship model between different mechanical performance and curve design variables is established.The optimization of variable-angle structure with mechanical properties as input variables is achieved.Meanwhile,a full-process angle-variable laying software platform from trajectory planning(CAD),trajectory optimization to manufacturing(CAM)is developed for facilitating the fiber placement application.

关键词： Fiber reinforced plastics Mechanical properties Robotic fiber placement Thermoplastics trajectory design Variable-angle laminate

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：