Learning dynamics of kernel-based deep neural networks in manifolds

作     者：Wei WU Xiaoyuan JING Wencai DU Guoliang CHEN Wei WU;Xiaoyuan JING;Wencai DU;Guoliang CHEN

作者机构：School of Computer Science Wuhan University Institute of Deep-sea Science and Engineering Chinese Academy of Sciences School of Computer Guangdong University of Petrochemical Technology Institute of Data Science City University of Macau College of Computer Science and Software Engineering Shenzhen University 

出 版 物：《Science China(Information Sciences)》 (中国科学:信息科学(英文版))

年 卷 期：2021年第64卷第11期

页      面：105-119页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by Key Project of National Natural Science Foundation of China (Grant No.61933013) Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDA22030301) NSFC-Key Project of General Technology Fundamental Research United Fund (Grant No. U1736211) Natural Science Foundation of Guangdong Province (Grant No. 2019A1515011076) Key Project of Natural Science Foundation of Hubei Province (Grant No.2018CFA024) 

主　　题：learning dynamics kernel-based convolution manifolds control model network stability 

摘      要：Convolutional neural networks(CNNs) obtain promising results via layered kernel convolution and pooling operations, yet the learning dynamics of the kernel remain obscure. We propose a continuous form to describe kernel-based convolutions through integration in neural manifolds. The status of spatial expression is proposed to analyze the stability of kernel-based CNNs. We divide CNN dynamics into the three stages of unstable vibration, collaborative adjusting, and stabilized fluctuation. According to the system control matrix of the kernel, the kernel-based CNN training proceeds via the unstable and stable status and is verified by numerical experiments.