Analysis and Experiments on Two Linear Discriminant Analysis Methods

作     者：Xu Yong~ 1,2 , Jin Zhong~ 2 , Yang Jingyu~2, Tang Zhengmin~2, Zhao Yingnan~3(1.Bio-Computing Research Center and Shenzhen Graduate School, Harbin Institute of Technology, Shenzhen 518055 2.Department of Computer Science & Technology, Nanjing University of Science & Technology,Nanjing 210094 3.College of Physics Science & Information Engineering, Jishou University, Jishou 416000) 

出 版 物：《Engineering Sciences》 (工程科学(英文版))

年 卷 期：2006年第3期

页      面：37-47页

学科分类：1305[艺术学-设计学（可授艺术学、工学学位）] 13[艺术学] 081104[工学-模式识别与智能系统] 08[工学] 0804[工学-仪器科学与技术] 081101[工学-控制理论与控制工程] 0811[工学-控制科学与工程] 

基　　金：The National Natural Science Foundation of China (Grant No.60472060  60473039 and 60472061) 

主　　题：Fisher criterion Foley-Sammon linear discriminant analysis(FSLDA) uncorrelated linear discriminant analysis(ULDA) correlation coefficient 

摘      要：Foley-Sammon linear discriminant analysis (FSLDA) and uncorrelated linear discriminant analysis (ULDA) are two well-known kinds of linear discriminant analysis. Both ULDA and FSLDA search the kth discriminant vector in an n-k+1 dimensional subspace, while they are subject to their respective constraints. Evidenced by strict demonstration, it is clear that in essence ULDA vectors are the covariance-orthogonal vectors of the corresponding eigen-equation. So, the algorithms for the covariance-orthogonal vectors are equivalent to the original algorithm of ULDA, which is time-consuming. Also, it is first revealed that the Fisher criterion value of each FSLDA vector must be not less than that of the corresponding ULDA vector by theory analysis. For a discriminant vector, the larger its Fisher criterion value is, the more powerful in discriminability it is. So, for FSLDA vectors, corresponding to larger Fisher criterion values is an advantage. On the other hand, in general any two feature components extracted by FSLDA vectors are statistically correlated with each other, which may make the discriminant vectors set at a disadvantageous position. In contrast to FSLDA vectors, any two feature components extracted by ULDA vectors are statistically uncorrelated with each other. Two experiments on CENPARMI handwritten numeral database and ORL database are performed. The experimental results are consistent with the theory analysis on Fisher criterion values of ULDA vectors and FSLDA vectors. The experiments also show that the equivalent algorithm of ULDA, presented in this paper, is much more efficient than the original algorithm of ULDA, as the theory analysis expects. Moreover, it appears that if there is high statistical correlation between feature components extracted by FSLDA vectors, FSLDA will not perform well, in spite of larger Fisher criterion value owned by every FSLDA vector. However, when the average correlation coefficient of feature components extracted by FSLDA ve