Exploring the Brain-like Properties of Deep Neural Networks:A Neural Encoding Perspective

作     者：Qiongyi Zhou Changde Du Huiguang He Qiongyi Zhou;Changde Du;Huiguang He

作者机构：Research Center for Brain-inspired Intelligence and National Laboratory of Pattern RecognitionInstitute of AutomationChinese Academy of SciencesBeijing 100190China School of Artificial IntelligenceUniversity of Chinese Academy of SciencesBeijing 100190China Center for Excellence in Brain Science and Intelligence TechnologyChinese Academy of SciencesBeijing 100190China 

出 版 物：《Machine Intelligence Research》 (机器智能研究（英文版）)

年 卷 期：2022年第19卷第5期

页      面：439-455页

核心收录：

学科分类：0831[工学-生物医学工程（可授工学、理学、医学学位）] 12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 080203[工学-机械设计及理论] 0835[工学-软件工程] 0802[工学-机械工程] 0836[工学-生物工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by National Natural Science Foundation of China(Nos.61976209 and 62020106015) the CAS International Collaboration Key Project,China(No.173211KYSB20190024) the Strategic Priority Research Program of CAS,China(No.XDB32040000) 

主　　题：Convolutional neural network(CNN) vision transformer(Vi T) multi-modal networks spatial-temporal networks visual neural encoding brain-like neural networks 

摘      要：Nowadays,deep neural networks(DNNs)have been equipped with powerful representation *** deep convolutional neural networks(CNNs)that draw inspiration from the visual processing mechanism of the primate early visual cortex have outperformed humans on object categorization and have been found to possess many brain-like ***,vision transformers(ViTs)have been striking paradigms of DNNs and have achieved remarkable improvements on many vision tasks compared to *** is natural to ask how the brain-like properties of ViTs *** the model paradigm,we are also interested in the effects of factors,such as model size,multimodality,and temporality,on the ability of networks to model the human visual pathway,especially when considering that existing research has been limited to *** this paper,we systematically evaluate the brain-like properties of 30 kinds of computer vision models varying from CNNs and ViTs to their hybrids from the perspective of explaining brain activities of the human visual cortex triggered by dynamic *** on two neural datasets demonstrate that neither CNN nor transformer is the optimal model paradigm for modelling the human visual *** reveal hierarchical correspondences to the visual pathway as CNNs ***,we find that multi-modal and temporal networks can better explain the neural activities of large parts of the visual cortex,whereas a larger model size is not a sufficient condition for bridging the gap between human vision and artificial *** study sheds light on the design principles for more brain-like *** code is available at https://***/QYiZhou/LWNeuralEncoding.