Learning to select the recombination operator for derivative-free optimization

作     者：Haotian Zhang Jianyong Sun Thomas Back Zongben Xu 

作者机构：School of Mathematics and StatisticsXi’an Jiaotong UniversityXi’an710049China Leiden Institute of Advanced Computer ScienceLeiden UniversityLeiden2333 CANetherlands 

出 版 物：《Science China Mathematics》 (中国科学（数学）（英文版）)

年 卷 期：2024年第67卷第6期

页      面：1457-1480页

核心收录：

学科分类：07[理学] 08[工学] 081104[工学-模式识别与智能系统] 070105[理学-运筹学与控制论] 0701[理学-数学] 0811[工学-控制科学与工程] 

基　　金：supported by National Natural Science Foundation of China(Grant No.62076197) Key Research and Development Project of Shaanxi Province(Grant No.2022GXLH-01-15)。 

主　　题：evolutionary algorithm differential evolution adaptive operator selection reinforcement learning deep learning 

摘      要：Extensive studies on selecting recombination operators adaptively,namely,adaptive operator selection(AOS),during the search process of an evolutionary algorithm(EA),have shown that AOS is promising for improving EA s performance.A variety of heuristic mechanisms for AOS have been proposed in recent decades,which usually contain two main components:the feature extraction and the policy setting.The feature extraction refers to as extracting relevant features from the information collected during the search process.The policy setting means to set a strategy(or policy)on how to select an operator from a pool of operators based on the extracted feature.Both components are designed by hand in existing studies,which may not be efficient for adapting optimization problems.In this paper,a generalized framework is proposed for learning the components of AOS for one of the main streams of EAs,namely,differential evolution(DE).In the framework,the feature extraction is parameterized as a deep neural network(DNN),while a Dirichlet distribution is considered to be the policy.A reinforcement learning method,named policy gradient,is used to train the DNN.As case studies,the proposed framework is applied to two DEs including the classic DE and a recently-proposed DE,which result in two new algorithms named PG-DE and PG-MPEDE,respectively.Experiments on the Congress of Evolutionary Computation(CEC)2018 test suite show that the proposed new algorithms perform significantly better than their counterparts.Finally,we prove theoretically that the considered classic methods are the special cases of the proposed framework.