Price-Based Residential Demand Response Management in Smart Grids:A Reinforcement Learning-Based Approach

作     者：Yanni Wan Jiahu Qin Xinghuo Yu Tao Yang Yu Kang Yanni Wan;Jiahu Qin;Xinghuo Yu;Tao Yang;Yu Kang

作者机构：Department of AutomationUniversity of Science and Technology of ChinaHefei 230027China Institute of Artificial IntelligenceHefei Comprehensive National Science CenterHefei 230088China School of EngineeringRMIT UniversityVIC 3000Australia State Key Laboratory of Synthetical Automation for Process IndustriesNortheastern UniversityShenyang 110819China Department of AutomationState Key Laboratory of Fire ScienceInstitute of Advanced TechnologyUniversity of Science and Technology of ChinaHefei 230027China Key Laboratory of Technology in Geo-Spatial Information Processing and Application SystemsChinese Academy of SciencesBeijing 100190China 

出 版 物：《IEEE/CAA Journal of Automatica Sinica》 (自动化学报（英文版）)

年 卷 期：2022年第9卷第1期

页      面：123-134页

核心收录：

学科分类：0810[工学-信息与通信工程] 1205[管理学-图书情报与档案管理] 12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 080802[工学-电力系统及其自动化] 0808[工学-电气工程] 08[工学] 081104[工学-模式识别与智能系统] 0802[工学-机械工程] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：This work was supported in part by the National Natural Science Foundation of China(61922076,61725304,61873252,61991403,61991400) in part by the Australian Research Council Discovery Program(DP200101199) 

主　　题：Demand response management(DRM) Markovian decision process(MDP) Monte Carlo simulation reinforcement learning(RL) smart grid 

摘      要：This paper studies price-based residential demand response management(PB-RDRM)in smart grids,in which non-dispatchable and dispatchable loads(including general loads and plug-in electric vehicles(PEVs))are both *** PB-RDRM is composed of a bi-level optimization problem,in which the upper-level dynamic retail pricing problem aims to maximize the profit of a utility company(UC)by selecting optimal retail prices(RPs),while the lower-level demand response(DR)problem expects to minimize the comprehensive cost of loads by coordinating their energy consumption *** challenges here are mainly two-fold:1)the uncertainty of energy consumption and RPs;2)the flexible PEVs’temporally coupled constraints,which make it impossible to directly develop a model-based optimization algorithm to solve the *** address these challenges,we first model the dynamic retail pricing problem as a Markovian decision process(MDP),and then employ a model-free reinforcement learning(RL)algorithm to learn the optimal dynamic RPs of UC according to the loads’*** proposed RL-based DR algorithm is benchmarked against two model-based optimization approaches(i.e.,distributed dual decomposition-based(DDB)method and distributed primal-dual interior(PDI)-based method),which require exact load and electricity price *** comparison results show that,compared with the benchmark solutions,our proposed algorithm can not only adaptively decide the RPs through on-line learning processes,but also achieve larger social welfare within an unknown electricity market environment.