AN INFORMATION THEORETICAL APPROACH TO NEURAL NETWORKS

作     者：Caro Lucas Caro Lucas Electrical Engineering Department, Tehran University, P.O. Box 14155/6181, Tehran IRAN

作者机构：Electrical Engineering Department Tehran University P. O. Box 14155/6181 Tehran IRAN 

出 版 物：《Systems Science and Mathematical Sciences》 (系统科学与数学(英文版))

年 卷 期：1993年第6卷第4期

页      面：353-372页

核心收录：

学科分类：0711[理学-系统科学] 12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 07[理学] 08[工学] 0811[工学-控制科学与工程] 081103[工学-系统工程] 

基　　金：This work was supported in part by Tehran University grant number 708 

主　　题：Neural networks stochastic systems information energy entropy 

摘      要：The purpose of this paper is to present a unified theory of several differentneural networks that have been proposed for solving various computation, pattern recog-nition, imaging, optimization, and other problems. The functioning of these networks ischaracterized by Lyapunov energy functions. The relationship between the deterministicand stochastic neural networks is examined. The simulated annealing methods for findingthe global optimum of an objective function as well as their generalization by injectingnoise into deterministic neural networks are discussed. A statistical interpretation of thedynamic evolution of the different neural networks is presented. The problem of trainingdifferent neural networks is investigated in this general framework. It is shown how thisapproach can be used not only for analyzing various neural networks, but also for the choiceof the proper neural network for solving any given problem and the design of a trainingalgorithm for the particular neural network.