Reinforcement Learning Based Obstacle Avoidance for Autonomous Underwater Vehicle

作     者：Prashant Bhopale Faruk Kazi Navdeep Singh 

作者机构：Electrical Engineering Department Veermata Jijabai Technological Institute 

出 版 物：《Journal of Marine Science and Application》 (船舶与海洋工程学报（英文版）)

年 卷 期：2019年第18卷第2期

页      面：228-238页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 08[工学] 

基　　金：the support of Centre of Excellence (CoE) in Complex and Nonlinear dynamical system (CNDS)  through TEQIP-II  VJTI  Mumbai  India 

主　　题：Obstacle avoidance Autonomous underwater vehicle Reinforcement learning Q-learning Function approximation 

摘      要：Obstacle avoidance becomes a very challenging task for an autonomous underwater vehicle(AUV)in an unknown underwater environment during exploration *** control in such case may be achieved using the model-based classical control techniques like PID and MPC but it required an accurate mathematical model of AUV and may fail due to parametric uncertainties,disturbance,or plant model *** the other hand,model-free reinforcement learning(RL)algorithm can be designed using actual behavior of AUV plant in an unknown environment and the learned control may not get affected by model uncertainties like a classical control *** model-based control model-free RL based controller does not require to manually tune controller with the changing environment.A standard RL based one-step Q-learning based control can be utilized for obstacle avoidance but it has tendency to explore all possible actions at given state which may increase number of *** a modified Q-learning based control approach is proposed to deal with these problems in unknown ***,function approximation is utilized using neural network(NN)to overcome the continuous states and large statespace problems which arise in RL-based controller *** proposed modified Q-learning algorithm is validated using MATLAB simulations by comparing it with standard Q-learning algorithm for single obstacle ***,the same algorithm is utilized to deal with multiple obstacle avoidance problems.