检索结果-南通市图书馆

Correction to a proposition related to ergodicity of Markov chains

在线全文

学校读者我要写书评

暂无评论

Frontiers of Mathematics in China 2021年第3期16卷 801-813页

作者： Ziqi TAN xianping guo School of Mathematics and Guangdong Province Key Laboratory of Computational Science Sun Yat-sen UniversityGuangzhou510275China

Proposition 5.5.6(ii)in the book Markov Chains and Stochastic Stability(2nd ed,Cambridge ***,2009)has been used in the proof of a theorem about ergodicity of Markov ***,an example in this paper shows that this proposi... 详细信息

关键词： Markov chain split chain petite set minorization condition

Convergence of Markov decision processes with constraints and state-action dependent discount factors

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China Mathematics 2020年第1期63卷 167-182页

作者： Xiao Wu xianping guo School of Mathematics and Statistics Zhaoqing UniversityZhaoqing 526061China School of Mathematics Sun Yat-sen UniversityGuangzhou 510275China

This paper is concerned with the convergence of a sequence of discrete-time Markov decision processes(DTMDPs)with constraints,state-action dependent discount factors,and possibly unbounded *** the convex analytic approach under mild conditions,we prove that the optimal values and optimal policies of the original DTMDPs converge to those of the"limit"***,we show that any countablestate DTMDP can be approximated by a sequence of finite-state DTMDPs,which are constructed using the truncation ***,we illustrate the approximation by solving a controlled queueing system numerically,and give the corresponding error bound of the approximation.

关键词： discrete-time Markov decision processes state-action dependent discount factors unbounded costs convergence

Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

在线全文

学校读者我要写书评

暂无评论

Science China Mathematics 2012年第11期55卷 2405-2416页

作者： ZHANG WenZhao guo xianping School of Mathematics and Computational Science Sun Yat-Sen UniversityGuangzhou 510275China

This paper attempts to study two-person nonzero-sum games for denumerable continuous-time Markov chains determined by transition rates,with an expected average *** transition rates are allowed to be unbounded,and the payoff functions may be unbounded from above and from *** give suitable conditions under which the existence of a Nash equilibrium is *** precisely,using the socalled "vanishing discount" approach,a Nash equilibrium for the average criterion is obtained as a limit point of a sequence of equilibrium strategies for the discounted criterion as the discount factors tend to *** results are illustrated with a birth-and-death game.

关键词： nonzero-sum game expected average criterion Nash equilibrium unbounded transition rates unbounded payoff function

First passage Markov decision processes with constraints and varying discount factors

在线全文

学校读者我要写书评

暂无评论

Frontiers of Mathematics in China 2015年第4期10卷 1005-1023页

作者： Xiao WU Xiaolong ZOU xianping guo School of Mathematics and Statistics Zhaoqing University Zhaoqing 526061 China School of Mathematics and Computational Science Sun Yat-sen UniversityGuangzhou 510275 China

This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an （infinite-dimensional） linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.

关键词： Discrete-time Markov decision process (DTMDP) constrainedoptimality varying discount factor unbounded cost

A minimization problem of the risk probability in first passage semi-Markov decision processes with loss rates

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China Mathematics 2015年第9期58卷 1923-1938页

作者： HUANG XiangXiang ZOU XiaoLong guo xianping School of Mathematics and Computational Science Sun Yat-Sen University

This paper is the first attempt to investigate the risk probability criterion in semi-Markov decision processes with loss rates. The goal is to find an optimal policy with the minimum risk probability that the total loss incurred during a first passage time to some target set exceeds a loss level. First, we establish the optimality equation via a successive approximation technique, and show that the value function is the unique solution to the optimality equation. Second, we give suitable conditions, under which we prove the existence of optimal policies and develop an algorithm for computing ?-optimal policies. Finally, we apply our main results to a business system.

关键词： semi-Markov decision processes loss rate risk probability first passage time optimal policy iteration algorithm

Optimal stopping time on discounted semi-Markov processes

在线全文

学校读者我要写书评

暂无评论

Frontiers of Mathematics in China 2021年第2期16卷 303-324页

作者： Fang CHEN xianping guo Zhong-Wei LIAO School of Mathematics Sun Yat-Sen UniversityGuangzhou 510275China College of Education for the Future Beijing Normal UniversityBeijing 100875China

This paper attempts to study the optimal stopping time for semi- Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov decision processes (SMDPs). The equivalence is embodied in the expected discounted cost functions of SMPs and SMDPs, that is, every stopping time of SMPs can induce a policy of SMDPs such that the value functions are equal, and vice versa. The existence of the optimal stopping time of SMPs is proved by this equivalence relation. Next, we give the optimality equation of the value function and develop an effective iterative algorithm for computing it. Moreover, we show that the optimal and ε-optimal stopping time can be characterized by the hitting time of the special sets. Finally, to illustrate the validity of our results, an example of a maintenance system is presented in the end.

关键词： Optimal stopping time semi-Markov processes(SMPs) value function semi-Markov decision processes(SMDPs) optimal policy iterative lgorithm

New optimality conditions for average-payoff continuous-time Markov games in Polish spaces

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China Mathematics 2011年第4期54卷 793-816页

作者： guo xianping HERNNDEZ-LERMA Onsimo The School of Mathematics and Computational Science Zhongshan UniversityGuangzhou 510275China Department of Mathematics CINVESTAV-IPN A.Postal 14-740Mxico D.F.07000Mxico

This paper concerns two-person zero-sum games for a class of average-payoff continuous-time Markov processes in Polish *** underlying processes are determined by transition rates that are allowed to be unbounded,and the payoff function may have neither upper nor lower *** use two optimality inequalities to replace the so-called optimality equation in the previous *** more general conditions,these optimality inequalities yield the existence of the value of the game and of a pair of optimal stationary *** some additional conditions we further establish the optimality equation ***,we use several examples to illustrate our results,and also to show the difference between the conditions in this paper and those in the *** particular,one of these examples shows that our approach is more general than all of the existing ones because it allows nonergodic Markov processes.

关键词： zero-sum stochastic Markov games average payoffs optimality inequalities unbounded transition rates

TOTAL REWARD CRITERIA FOR UNCONSTRAINED/CONSTRAINED CONTINUOUS-TIME MARKOV DECISION PROCESSES

在线全文

学校读者我要写书评

暂无评论

Journal of Systems Science & Complexity 2011年第3期24卷 491-505页

作者： xianping guo Lanlan ZHANG School of Mathematics and Computational Science Sun Yat-sen University Guangzhou 510275 China School of Public Health and Tropical Medicine Southern Medical University Guangzhou 510515 China

This paper studies denumerable continuous-time Markov decision processes with expected total reward criteria. The authors first study the unconstrained model with possible unbounded transition rates, and give suitable conditions on the controlled system＇s primitive data under which the authors show the existence of a solution to the total reward optimality equation and also the existence of an optimal stationary policy. Then, the authors impose a constraint on an expected total cost, and consider the associated constrained model. Basing on the results about the unconstrained model and using the Lagrange multipliers approach, the authors prove the existence of constrained-optimal policies under some additional conditions. Finally, the authors apply the results to controlled queueing systems.

关键词： Constrained-optimal policy continuous-time Markov decision process optimal policy total reward criterion unbounded reward/cost and transition rates.

在线全文

学校读者我要写书评

暂无评论

非平稳MDP—平均样本轨道最优

数学物理学报（A辑） 2000年第1期20卷 31-35页

作者：郭先平中山大学数学系广州510275

作者考虑的是任意状态空间，任意行动空间非平稳MDP的平均样本轨道目标．在弱遍历条件下，用鞅的极限理论，证明了最优马氏策略的存在性，推广了***，***，***，***，***[1]（1993）的主要结果．

关键词：马氏决策规划平均样本轨道目标非平衡 MDP