Clique-based Cooperative Multiagent Reinforcement Learning Using Factor Graphs

作     者：Zhen Zhang Dongbin Zhao 

作者机构：the State Key Laboratory of Management and Control for Complex SystemsInstitute of AutomationChinese Academy of Sciences Department of Electric EngineeringCollege of Automation EngineeringQingdao University 

出 版 物：《IEEE/CAA Journal of Automatica Sinica》 (自动化学报（英文版）)

年 卷 期：2014年第1卷第3期

页      面：248-256页

核心收录：

学科分类：0810[工学-信息与通信工程] 1205[管理学-图书情报与档案管理] 08[工学] 081203[工学-计算机应用技术] 0802[工学-机械工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China  (61273136  6103400) 

主　　题：Sensors Games Learning (artificial intelligence) Approximation algorithms Algorithm design and analysis Heuristic algorithms Sparse matrices 

摘      要：In this paper,we propose a clique-based sparse reinforcement learning(RL) algorithm for solving cooperative tasks.The aim is to accelerate the learning speed of the original sparse RL algorithm and to make it applicable for tasks decomposed in a more general manner.First,a transition function is estimated and used to update the Q-value function,which greatly reduces the learning time.Second,it is more reasonable to divide agents into cliques,each of which is only responsible for a specific subtask.In this way,the global Q-value function is decomposed into the sum of several simpler local Q-value functions.Such decomposition is expressed by a factor graph and exploited by the general maxplus algorithm to obtain the greedy joint action.Experimental results show that the proposed approach outperforms others with better performance.