Energy-information trade-off induces continuous and discontinuous phase transitions in lateral predictive coding

作     者：Zhen-Ye Huang Ruyi Zhou Miao Huang Hai-Jun Zhou 

作者机构：Key Laboratory for Theoretical PhysicsInstitute of Theoretical PhysicsChinese Academy of SciencesBeijing 100190China School of Physical SciencesUniversity of Chinese Academy of SciencesBeijing 100049China School of Optics and PhotonicsBeijing Institute of TechnologyBeijing 100081China Minjiang Collaborative Center for Theoretical PhysicsMinjiang UniversityFuzhou 350108China 

出 版 物：《Science China(Physics,Mechanics & Astronomy)》 (中国科学：物理学、力学、天文学（英文版）)

年 卷 期：2024年第67卷第6期

页      面：79-85页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Natural Science Foundation of China(Grant Nos.12047503,11747601 and 12247104) the National Innovation Institute of Defense Technology(Grant No.22TQ0904ZT01025) 

主　　题：predictive coding recurrent neural network phase transition internal model free energy 

摘      要：Lateral predictive coding is a recurrent neural network that creates energy-efficient internal representations by exploiting statistical regularity in sensory ***,we analytically investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding and numerically minimize a free energy *** observed several phase transitions in the synaptic weight matrix,particularly a continuous transition that breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory *** optimal network follows an ideal gas law over an extended temperature range and saturates the efficiency upper bound of energy *** results provide theoretical insights into the emergence and evolution of complex internal models in predictive processing systems.