检索结果-南通市图书馆

A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems

在线全文

学校读者我要写书评

暂无评论

Science China Chemistry 2015年第12期58卷 143-157页

作者： WEI QingLai LIU DeRong State Key Laboratory of Management and Control for Complex Systems Institute of AutomationChinese Academy of Sciences School of Automation and Electrical Engineering University of Science and Technology Beijing

In this paper, a novel iterative Q-learning algorithm, called "policy iteration based deterministic Qlearning algorithm", is developed to solve the optimal control problems for discrete-time deterministic nonlinear systems. The idea is to use an iterative adaptive dynamic programming(ADP) technique to construct the iterative control law which optimizes the iterative Q function. When the optimal Q function is obtained, the optimal control law can be achieved by directly minimizing the optimal Q function, where the mathematical model of the system is not necessary. Convergence property is analyzed to show that the iterative Q function is monotonically non-increasing and converges to the solution of the optimality equation. It is also proven that any of the iterative control laws is a stable control law. Neural networks are employed to implement the policy iteration based deterministic Q-learning algorithm, by approximating the iterative Q function and the iterative control law, respectively. Finally, two simulation examples are presented to illustrate the performance of the developed algorithm.

关键词： adaptive critic designs adaptive dynamic programming approximate dynamic programming Q-learning policy iteration neural networks nonlinear systems optimal control

Optimal synchronization control formulti-agent systems with input saturation:a nonzero-sum game

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Information Technology & Electronic Engineering 2022年第7期23卷 1010-1019页

作者： Hongyang LI Qinglai WEI School of Artificial Intelligence University of Chinese Academy of SciencesBeijing 100049China The State Key Laboratory for Management and Control of Complex Systems Institute of AutomationChinese Academy of SciencesBeijing 100190China Institute of Systems Engineering Macao University of Science and TechnologyMacao 999078China

This paper presents a novel optimal synchronization control method for multi-agent systems with input *** multi-agent game theory is introduced to transform the optimal synchronization control problem into a multi-agent nonzero-sum ***,the Nash equilibrium can be achieved by solving the coupled Hamilton–Jacobi–Bellman(HJB)equations with nonquadratic input energy terms.A novel off-policy reinforcement learning method is presented to obtain the Nash equilibrium solution without the system models,and the critic neural networks(NNs)and actor NNs are introduced to implement the presented *** analysis is provided,which shows that the iterative control laws converge to the Nash *** results show the good performance of the presented method.

关键词： Optimal synchronization control Multi-agent systems Nonzero-sum game adaptive dynamic programming Input saturation Off-policy reinforcement learning Policy iteration

Minimax Q-learning design for H_(∞) control of linear discrete-time systems

在线全文

学校读者我要写书评

暂无评论

Frontiers of Information Technology & Electronic Engineering 2022年第3期23卷 438-451页

作者： Xinxing LI Lele XI Wenzhong ZHA Zhihong PENG Information Science Academy China Electronics Technology Group CorporationBeijing 100086China School of Automation Beijing Institute of TechnologyBeijing 100081China Peng Cheng Laboratory Shenzhen 518052China

The H_(∞)control method is an effective approach for attenuating the effect of disturbances on practical systems, but it is difficult to obtain the H_(∞)controller due to the nonlinear Hamilton-Jacobi-Isaacs equation, even for linear systems. This study deals with the design of an H_(∞)controller for linear discrete-time systems. To solve the related game algebraic Riccati equation(GARE), a novel model-free minimax Q-learning method is developed, on the basis of an offline policy iteration algorithm, which is shown to be Newton’s method for solving the GARE. The proposed minimax Q-learning method, which employs off-policy reinforcement learning, learns the optimal control policies for the controller and the disturbance online, using only the state samples generated by the implemented behavior policies. Different from existing Q-learning methods, a novel gradient-based policy improvement scheme is proposed. We prove that the minimax Q-learning method converges to the saddle solution under initially admissible control policies and an appropriate positive learning rate, provided that certain persistence of excitation(PE)conditions are satisfied. In addition, the PE conditions can be easily met by choosing appropriate behavior policies containing certain excitation noises, without causing any excitation noise bias. In the simulation study, we apply the proposed minimax Q-learning method to design an H_(∞)load-frequency controller for an electrical power system generator that suffers from load disturbance, and the simulation results indicate that the obtained H_(∞)load-frequency controller has good disturbance rejection performance.

关键词： H_(∞)control Zero-sum dynamic game Reinforcement learning adaptive dynamic programming Minimax Q-learning Policy iteration

Data⁃Based Feedback Relearning Algorithm for Robust Control of SGCMG Gimbal Servo System with Multi⁃source Disturbance

在线全文

学校读者我要写书评

暂无评论

Transactions of Nanjing University of Aeronautics and Astronautics 2021年第2期38卷 225-236页

作者： ZHANG Yong MU Chaoxu LU Ming School of Electrical and Information Engineering Tianjin UniversityTianjin 300072P.R.China Beijing Institute of Control Engineering Beijing 100190P.R.China

Single gimbal control moment gyroscope(SGCMG)with high precision and fast response is an important attitude control system for high precision docking,rapid maneuvering navigation and guidance system in the aerospace *** this paper,considering the influence of multi-source disturbance,a data-based feedback relearning(FR)algorithm is designed for the robust control of SGCMG gimbal servo *** on adaptive dynamic programming and least-square principle,the FR algorithm is used to obtain the servo control strategy by collecting the online operation data of SGCMG *** is a model-free learning strategy in which no prior knowledge of the SGCMG model is ***,combining the reinforcement learning mechanism,the servo control strategy is interacted with system dynamic of *** adaptive evaluation and improvement of servo control strategy against the multi-source disturbance are ***,a data redistribution method based on experience replay is designed to reduce data correlation to improve algorithm stability and data utilization ***,by comparing with other methods on the simulation model of SGCMG,the effectiveness of the proposed servo control strategy is verified.

关键词： control moment gyroscope feedback relearning algorithm servo control reinforcement learning multisource disturbance adaptive dynamic programming

Optimal Fault Tolerant Control of Reconfigurable Manipulator with Actuator Saturation

在线全文

学校读者我要写书评

暂无评论

Optimal Fault Tolerant Control of Reconfigurable Manipulator...

第32届中国控制与决策会议

作者： Fujie Nie Fan Zhou Bo Dong Tianjiao An Yuanchun Li Department of Control Science and Engineering Changchun University of Technology

In this paper, an optimal active fault tolerant control（FTC） method is presented for reconfigurable manipulator with actuator saturation based on the adaptive dynamic programming（ADP） approach. First, the saturation constraints of the manipulator systems are tackled by using hyperbolic tangent functions. Second, an actuator fault function, which is estimated via a fault observer, is used to construct the performance index function. Then, the solution of Hamiltonian-Jacobi-Bellman（HJB） equation is solved by using ADP and policy iteration（PI） methods, and the optimal performance index function is approximated by establishing a critic neural network（NN）. Based on the Lyapunov theory, the closed-loop manipulator system is proved to be asymptotic stable under the optimal FTC system. Finally, simulation results are demonstrated the validity of the proposed scheme.

关键词： adaptive dynamic programming reconfigurable manipulators actuator saturation optimal fault tolerant control neural network

A Neural Critic Learning Method for Nonlinear Decentralized Stabilization with Unknown Perturbation

在线全文

cnki会议

学校读者我要写书评

暂无评论

A Neural Critic Learning Method for Nonlinear Decentralized ...

第31届中国过程控制会议（CPCC2020）

作者： HUO Yu WANG Ding QIAO Junfei Faculty of Information Technology Beijing University of Technology Beijing Key Laboratory of Computational Intelligence and Intelligent System Beijing University of Technology

In order to realize the robust decentralized stabilization of continuous-time nonlinear systems with general uncertainties, a neural-network-based online optimal control strategy is established, where the solution of the corresponding Hamilton-Jacobi-Bellman(HJB) equation is obtained. Specifically, as a necessary part of the control system, a neural network is involved with strong online learning ability. The novelty of this paper is that the well-known adaptive dynamic programming method is extended to deal with the nonlinear feedback control problem under uncertain and large-scale environment. First, the robust decentralized stabilization problem is transformed into the decentralized optimal control design problem. On this basis, a critic neural network is constructed to solve the modified HJB equation corresponding to the nominal system in an online form. Moreover, a novel updating rule compared to the traditional adaptive critic algorithms is proposed, which reduces the requirement of the initial stabilizing controller and brings in particular reinforcements to the adaptive critic control design. Finally, an example is included to show the feasibility and effectiveness of the obtained theoretical results.

关键词： adaptive dynamic programming dynamical uncertainty Neural networks Optimal control Robust decentralized stabilization

Neuro-Control for Continuous-Time Stochastic Nonlinear Systems via Online Policy Iteration Algorithm

在线全文

cnki会议

学校读者我要写书评

暂无评论

Neuro-Control for Continuous-Time Stochastic Nonlinear Syste...

第32届中国控制与决策会议

作者： Tianmin Zhou Jiaxu Hou HANDong Li Zengru Di Bo Zhao School of Systems Science Beijing Normal University

This paper is concerned with the neuro-control for continuous-time nonlinear systems subject to stochastic *** to the stochastic disturbance,the traditional value function in existing literature cannot meet the stochastic control problems,since mixed second partial derivatives are employed to construct modified value function of conditional *** solve the Hamilton-Jacobi-Bellman equation,a novel online policy iteration algorithm with an Ito correction term is developed with establishing a critic neural network to approximate the optimal value ***,the online optimal control can be obtained in a closed-loop *** closed-loop system is guaranteed to be stable in probability via Lyapunov’s direct ***,numerical example is provided to illustrate the effectiveness of the developed control method.

关键词： adaptive dynamic programming Reinforcement Learning Policy Iteration Stochastic Nonlinear

Event-Driven Optimal Regulation with SNAC Framework for Constrained Nonlinear Plants

在线全文

cnki会议

学校读者我要写书评

暂无评论

Event-Driven Optimal Regulation with SNAC Framework for Cons...

第40届中国控制会议

作者： Lingzhi Hu Ding Wang Mingming Zhao Jin Ren Faculty of Information Technology Beijing University of Technology Beijing Key Laboratory of Computational Intelligence and Intelligent System Bejing University of Technology Beijing Institute of Artificial Intelligence Beijing University of Technology

In order to reduce the computational burden effectively and ensure that the system has good control performance, an event-driven near-optimal control algorithm is developed for discrete-time nonlinear systems with input constraints. A single network adaptive critic algorithm is adopted to address the constrained optimal control problem. First, the optimal cost function and the optimal control strategy are obtained theoretically through the iterative algorithm. Then, an appropriate triggering threshold is established, so that the system satisfies the event-driven mechanism. In addition, the action network of classical adaptive dynamic programming is omitted. The derivative of the approximate cost function is the output of the critic network and then is used to obtain the approximate optimal control law. Furthermore, the convergence analysis of the iterative algorithm is proved. Finally, a simulation example is conducted to verify the effectiveness of the proposed scheme.

关键词： adaptive dynamic programming Control Constraints Discrete-Time Systems Event-Driven Control Neural Networks Single Network adaptive Critic

Value-Iteration-Based Neuro-Optimal Tracking Control for Affine Systems with Completely Unknown dynamics

在线全文

cnki会议

学校读者我要写书评

暂无评论

Value-Iteration-Based Neuro-Optimal Tracking Control for Aff...

第三十九届中国控制会议

作者： Mingming Ha Ding Wang Derong Liu School of Automation and Electrical Engineering University of Science and Technology Beijing Faculty of Information Technology Beijing University of Technology Beijing Key Laboratory of Computational Intelligence and Intelligent System Beijing University of Technology School of Automation Guangdong University of Technology

In this paper, the versatile value-iteration-based control method, aimed at affine systems with unknown dynamics, is proposed to deal with the optimal tracking control problem. Neural networks are adopted to approximate system dynamics and a novel approach is presented to estimate the steady state control input based on the established identifier. Additionally, two other neural networks, called the critic network and the action network, are used to implement the optimal tracking control ***, based on the proposed method, the tracking controller is designed to control a specific simulation example. It is shown that, for any randomly given initial state vector, the controller is able to make the affine system track the reference trajectory without knowing the system dynamics.

关键词： adaptive dynamic programming Neuro-Optimal Tracking Control Value Iteration Affine Systems with Unknown dynamics