Reinforcement learning-based scheduling of multi-battery energy storage system

作     者：CHENG Guangran DONG Lu YUAN Xin SUN Changyin CHENG Guangran;DONG Lu;YUAN Xin;SUN Changyin

作者机构：School of AutomationSoutheast UniversityNanjing 210096China Peng Cheng LaboratoryShenzhen 518066China School of Cyber Science and EngineeringSoutheast UniversityNanjing 211189China 

出 版 物：《Journal of Systems Engineering and Electronics》 (系统工程与电子技术（英文版）)

年 卷 期：2023年第34卷第1期

页      面：117-128页

核心收录：

学科分类：080801[工学-电机与电器] 0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Key R&D Program of China (2018AAA0101400) the National Natural Science Foundation of China (61921004,62173251,U1713209,62236002) the Fundamental Research Funds for the Central Universities Guangdong Provincial Key Laboratory of Intelligent Decision and Cooperative Control 

主　　题：multi-battery energy storage system(MBESS) reinforcement learning periodic value iteration data-driven 

摘      要：In this paper, a reinforcement learning-based multibattery energy storage system(MBESS) scheduling policy is proposed to minimize the consumers ’ electricity cost. The MBESS scheduling problem is modeled as a Markov decision process(MDP) with unknown transition probability. However, the optimal value function is time-dependent and difficult to obtain because of the periodicity of the electricity price and residential load. Therefore, a series of time-independent action-value functions are proposed to describe every period of a day. To approximate every action-value function, a corresponding critic network is established, which is cascaded with other critic networks according to the time sequence. Then, the continuous management strategy is obtained from the related action network. Moreover, a two-stage learning protocol including offline and online learning stages is provided for detailed implementation in real-time battery management. Numerical experimental examples are given to demonstrate the effectiveness of the developed algorithm.