Hybrid Q-learning for data-based optimal control of non-linear switching system

作     者：LI Xiaofeng DONG Lu SUN Changyin LI Xiaofeng;DONG Lu;SUN Changyin

作者机构：School of AutomationSoutheast UniversityNanjing 210096China School of Artificial IntelligenceAnhui UniversityHefei 230601China School of Cyber Science and EngineeringSoutheast UniversityNanjing 211189China 

出 版 物：《Journal of Systems Engineering and Electronics》 (系统工程与电子技术（英文版）)

年 卷 期：2022年第33卷第5期

页      面：1186-1194页

核心收录：

学科分类：1201[管理学-管理科学与工程(可授管理学、工学学位)] 0808[工学-电气工程] 07[理学] 070105[理学-运筹学与控制论] 0701[理学-数学] 0811[工学-控制科学与工程] 

基　　金：supported by the National Key R&D Program of China(2018AAA0101400) the Natural Science Foundation of Jiangsu Province of China(BK20202006) the National Natural Science Foundation of China(61921004,62173251). 

主　　题：switching system hybrid action space optimal control reinforcement learning hybrid Q-learning(HQL) 

摘      要：In this paper,the optimal control of non-linear switching system is investigated without knowing the system dynamics.First,the Hamilton-Jacobi-Bellman(HJB)equation is derived with the consideration of hybrid action space.Then,a novel data-based hybrid Q-learning(HQL)algorithm is proposed to find the optimal solution in an iterative manner.In addition,the theoretical analysis is provided to illustrate the convergence and optimality of the proposed algorithm.Finally,the algorithm is implemented with the actor-critic(AC)structure,and two linear-in-parameter neural networks are utilized to approximate the functions.Simulation results validate the effectiveness of the data-driven method.