Robotic Knee Tracking Control to Mimic the Intact Human Knee Profile Based on Actor-Critic Reinforcement Learning

作     者：Ruofan Wu Zhikai Yao Jennie Si He(Helen)Huang Ruofan Wu;Zhikai Yao;Jennie Si;He(Helen)Huang

作者机构：School of ElectricalComputer and Energy EngineeringArizona State UniversityTempeAZ 85287 USA Department of Biomedical EngineeringNorth Carolina State UniversityRaleighNC 27695 USA University of North Carolina at Chapel HillChapel HillNC 27599 USA 

出 版 物：《IEEE/CAA Journal of Automatica Sinica》 (自动化学报（英文版）)

年 卷 期：2022年第9卷第1期

页      面：19-30页

核心收录：

学科分类：080202[工学-机械电子工程] 08[工学] 0804[工学-仪器科学与技术] 0802[工学-机械工程] 0811[工学-控制科学与工程] 

基　　金：This work was partly supported by the National Science Foundation(1563921 1808752 1563454 1808898) 

主　　题：Automatic tracking of intact knee configuration of robotic knee prosthesis direct heuristic dynamic programming(dHDP) reinforcement learning control 

摘      要：We address a state-of-the-art reinforcement learning(RL)control approach to automatically configure robotic pros-thesis impedance parameters to enable end-to-end,continuous locomotion intended for transfemoral amputee ***,our actor-critic based RL provides tracking control of a robotic knee prosthesis to mimic the intact knee *** is a significant advance from our previous RL based automatic tuning of prosthesis control parameters which have centered on regulation control with a designer prescribed robotic knee profile as the *** addition to presenting the tracking control algorithm based on direct heuristic dynamic programming(dHDP),we provide a control performance guarantee including the case of constrained *** show that our proposed tracking control possesses several important properties,such as weight convergence of the learning networks,Bellman(sub)optimality of the cost-to-go value function and control input,and practical stability of the human-robot *** further provide a systematic simulation of the proposed tracking control using a realistic human-robot system simulator,the OpenSim,to emulate how the dHDP enables level ground walking,walking on different terrains and at different *** results show that our proposed dHDP based tracking control is not only theoretically suitable,but also practically useful.