检索结果-南通市图书馆

A STOCHASTIC TRUST-REGION FRAMEWORK FOR POLICY OPTIMIZATION

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computational Mathematics 2022年第6期40卷 1004-1030页

作者： Mingming Zhao Yongfeng Li zaiwen wen Beijing International Center for Mathematical Research Peking UniversityBeijing 100871China

In this paper,we study a few challenging theoretical and numerical issues on the well known trust region policy optimization for deep reinforcement learning.The goal is to find a policy that maximizes the total expected reward when the agent acts according to the policy.The trust region subproblem is constructed with a surrogate function coherent to the total expected reward and a general distance constraint around the latest policy.We solve the subproblem using a preconditioned stochastic gradient method with a line search scheme to ensure that each step promotes the model function and stays in the trust region.To overcome the bias caused by sampling to the function estimations under the random settings,we add the empirical standard deviation of the total expected reward to the predicted increase in a ratio in order to update the trust region radius and decide whether the trial point is accepted.Moreover,for a Gaussian policy which is commonly used for continuous action space,the maximization with respect to the mean and covariance is performed separately to control the entropy loss.Our theoretical analysis shows that the deterministic version of the proposed algorithm tends to generate a monotonic improvement of the total expected reward and the global convergence is guaranteed under moderate assumptions.Comparisons with the state-of-the–art methods demonstrate the effectiveness and robustness of our method over robotic controls and game playings from OpenAI Gym.

关键词： Deep reinforcement learning Stochastic trust region method Policy optimization Global convergence Entropy control

On the local convergence of a stochastic semismooth Newton method for nonsmooth nonconvex optimization

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China Mathematics 2022年第10期65卷 2151-2170页

作者： Andre Milzarek Xiantao Xiao zaiwen wen Michael Ulbrich School of Data Science The Chinese University of Hong KongShenzhenShenzhen 518172China Shenzhen Research Institute of Big Data Shenzhen 518172China Shenzhen Institute of Arti cial Intelligence and Robotics for Society Shenzhen 518172China School of Mathematical Sciences Dalian University of TechnologyDalian 116024China Beijing International Center for Mathematical Research Peking UniversityBeijing 100871China Department of Mathematics Technical University of MunichGarching bei Munchen 85748Germany

In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth convex terms in the objective function.We assume that the gradient and Hessian information of the smooth part of the objective function can only be approximated and accessed via calling stochastic firstand second-order oracles.The approach combines stochastic semismooth Newton steps,stochastic proximal gradient steps and a globalization strategy based on growth conditions.We present tail bounds and matrix concentration inequalities for the stochastic oracles that can be utilized to control the approximation errors via appropriately adjusting or increasing the sampling rates.Under standard local assumptions,we prove that the proposed algorithm locally turns into a pure stochastic semismooth Newton method and converges r-linearly or r-superlinearly with high probability.

关键词： nonsmooth stochastic optimization stochastic approximation semismooth Newton method stochastic second-order information local convergence

A GENERAL TWO-LEVEL SUBSPACE METHOD FOR NONLINEAR OPTIMIZATION

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Journal of Computational Mathematics 2018年第6期36卷 881-902页

作者： Cheng Chen zaiwen wen Yaxiang Yuan University of Chinese Academy of Sciences Beijing 100190 China LSEC ICMSEC Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing China Beijing International Center for Mathematical Research Peking University Beijing 100871 China LSEC ICMSEC Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing 100190 China

A new two-level subspace method is proposed for solving the general unconstrained minimization formulations discretized from infinite-dimensional optimization problems. At each iteration, the algorithm executes either a direct step on the current level or a coarse subspace correction step. In the coarse subspace correction step, we augment the traditional coarse grid space by a two-dimensional subspace spanned by the coordinate direction and the gradient direction at the current point. Global convergence is proved and convergence rate is studied under some mild conditions on the discretized functions. Preliminary numerical experiments on a few variational problems show that our two-level subspace method is promising.

关键词： Nonlinear optimization Convex and nonconvex problems Subspace technique Multigrid/multilevel method Large-scale problems

EXTENDED LEVENBERG-MARQUARDT METHOD FOR COMPOSITE FUNCTION MINIMIZATION

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computational Mathematics 2017年第4期35卷 529-546页

作者： Jianchao Huang zaiwen wen Xiantao Xiao School of Mathematical Sciences Shanghai Jiao Tong University Shanghai 200240 China Beijing International Center for Mathematical Research Peking University Beijing 100871 China School of Mathematical Sciences Dalian University of Technology Dalian 116024 China

In this paper, we propose an extended Levenberg-Marquardt （ELM） framework that generalizes the classic Levenberg-Marquardt （LM） method to solve the unconstrained minimization problem min ρ（r（x））, where r ： Rn→ Rm and ρ ： Rm → R. We also develop a few inexact variants which generalize ELM to the cases where the inner subproblem is not solved exactly and the Jaeobian is simplified, or perturbed. Global convergence and local superlinear convergence are established under certain suitable conditions. Numerical results show that our methods are promising.

关键词： Unconstrained minimization Composite function Levenberg-Marquardt method.

BLOCK ALGORITHMS WITH AUGMENTED RAYLEIGH-RITZ PROJECTIONS FOR LARGE-SCALE EIGENPAIR COMPUTATION

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computational Mathematics 2019年第6期37卷 889-915页

作者： Haoyang Liu zaiwen wen Chao Yang Yin Zhang Beijing International Center for Mathematical Research Peking UniversityBeijing 100871China Computational Research Division Lawrence Berkeley National LaboratoryBerkeleyCAUSA Department of Computational and Applied Mathematics Rice UniversityHoustonUSA

Most iterative algorithms for eigenpair computation consist of two main steps:a subspace update(SU)step that generates bases for approximate eigenspaces,followed by a Rayleigh-Ritz(RR)projection step that extracts approximate eigenpairs.So far the predominant methodology for the SU step is based on Krylov subspaces that builds orthonormal bases piece by piece in a sequential manner.In this work,we investigate block methods in the SU step that allow a higher level of concurrency than what is reachable by Krylov subspace methods.To achieve a competitive speed,we propose an augmented Rayleigh-Ritz(ARR)procedure.Combining this ARR procedure with a set of polynomial accelerators,as well as utilizing a few other techniques such as continuation and deflation,we construet a block algorithm designed to reduce the number of RR steps and elevate concurrency in the SU steps.Extensive computational experiments are conducted in C on a representative set of test problems to evaluate the performance of two variants of our algorithm.Numerical results,obtained on a many-core computer without explicit code parallelization,show that when computing a relatively large number of eigenpairs,the performance of our algorithms is competitive with that of several state-of-the-art eigensolvers.

关键词： Extreme eigenpairs Augmented Rayleigh-Ritz projection

An alternating direction algorithm for matrix completion with nonnegative factors

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Mathematics in China 2012年第2期7卷 365-384页

作者： Yangyang XU Wotao YIN zaiwen wen Yin ZHANG Department of Computational and Applied Mathematics Rice University Houston TX 77005 USA Department of Mathematics and Institute of Natural Sciences Shanghai Jiao Tong University Shanghai 200240 China

This paper introduces an algorithm for the nonnegative matrix factorization-and-completion problem, which aims to find nonnegative low-rank matrices X and Y so that the product XY approximates a nonnegative data matrix M whose elements are partially known （to a certain accuracy）. This problem aggregates two existing problems：（i） nonnegative matrix factorization where all entries of M are given, and （ii） low-rank matrix completion where non- negativity is not required. By taking the advantages of both nonnegativity and low-rankness, one can generally obtain superior results than those of just using one of the two properties. We propose to solve the non-convex constrained least-squares problem using an algorithm based on tile classical alternating direction augmented Lagrangian method. Preliminary convergence properties of the algorithm and numerical simulation results are presented. Compared to a recent algorithm for nonnegative matrix factorization, the proposed algorithm produces factorizations of similar quality using only about half of the matrix entries. On tasks of recovering incomplete grayscale and hyperspeetral images, the proposed algorithm yields overall better qualities than those produced by two recent matrix-completion algorithms that do not exploit nonnegativity.

关键词： nonnegative matrix factorization matrix completion alternating direction method hyperspectral unmixing

Joint Bandwidth Allocation and Path Selection in WANs with Path Cardinality Constraints

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Communications and Information Networks 2021年第3期6卷 237-250页

作者： Jinxin Wang Fan Zhang Zhonglin Xie zaiwen wen Gong Zhang Department of Systems Engineering and Engineering Management CUHKHong Kong 999077China Theory Lab 2012 Labs.Huawei Technologies Co.LtdHong Kong 999077China School of Mathematical Science Peking UniversityBeijing 100871China Beijing International Center for Mathematical Research Peking UniversityBeijing 100871China

In this paper,we study the joint bandwidth allocation and path selection problem,which is an extension of the well-known network utility maximization(NUM)problem,via solving a multi-objective minimization problem under path cardinality constraints.Specifically,such a problem formulation captures various types of objectives including proportional fairness,average delay,as well as load balancing.In addition,in order to handle the"unsplittable flows",path cardinality constraints are added,making the resulting optimization problem quite challenging to solve due to intrinsic nonsmoothness and nonconvexity.Almost all existing works deal with such a problem using relaxation techniques to transform it into a convex optimization problem.However,we provide a novel solution framework based on the linearized alternating direction method of multipliers(LADMM)to split the original problem with coupling terms into several subproblems.We then derive that these subproblems,albeit nonconvex nonsmooth,are actually simple to solve and easy to implement,which can be of independent interest.Under some mild assumptions,we prove that any limiting point of the generated sequence of the proposed algorithm is a stationary point.Numerical simulations are performed to demonstrate the advantages of our proposed algorithm compared with various baselines.

关键词： bandwidth allocation unsplittable flows cardinality constraints network utility maximization LADMM

LINEARLY CONVERGENT FIRST-ORDER ALGORITHMS FOR SEMIDEFINITE PROGRAMMING

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computational Mathematics 2017年第4期35卷 452-468页

作者： Cong D. Dang Guanghui Lan zaiwen wen Department of Industrial and Systems Engineering University of Florida Gainesville FL 32611 H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of TechnologyAtlanta GA 30332 USA Beijing International Center for Mathematical Research Peking University Beijing 100871 China

In this paper, we consider two different formulations （one is smooth and the other one is nonsmooth） for solving linear matrix inequalities （LMIs）, an important class of semidefinite programming （SDP）, under a certain Slater constraint qualification assumption. We then propose two first-order methods, one based on subgradient method and the other based on Nesterov＇s optimal method, and show that they converge linearly for solving these formulations. Moreover, we introduce an accelerated prox-level method which converges linearly uniformly for both smooth and non-smooth problems without requiring the input of any problem parameters. Finally, we consider a special case of LMIs, i.e., linear system of inequalities, and show that a linearly convergent algorithm can be obtained under a much weaker assumption.

关键词： Semi-definite Programming Linear Matrix Inequalities Error Bounds Linear Convergence

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Preface

Communications in Mathematics and Statistics 2023年第1期11卷 1-2页

作者： Weinan E Princeton Falai Chen zaiwen wen University&Peking University University of Science and Technology of China Peking University

In the past several centuries,there have been two major different paradigms for doing scientific research:the Keplerian paradigm and the Newtonian paradigm.In the Keplerian paradigm,natural laws are discovered through the analysis of data while in the Newtonian paradigm,one focuses on pursuing fundamental principles that govern Nature.

关键词： Newtonian principles fundamental