Degrees of freedom in low rank matrix estimation

作     者：YUAN Ming 

作者机构：Department of Statistics University of Wisconsin-Madison 

出 版 物：《Science China Mathematics》 (中国科学：数学（英文版）)

年 卷 期：2016年第59卷第12期

页      面：2485-2502页

核心收录：

学科分类：02[经济学] 0202[经济学-应用经济学] 020208[经济学-统计学] 07[理学] 0714[理学-统计学（可授理学、经济学学位）] 070103[理学-概率论与数理统计] 0701[理学-数学] 

基　　金：supported by National Science Foundation of USA (Grant No. DMS1265202) National Institutes of Health of USA (Grant No. 1-U54AI117924-01) 

主　　题：degrees of freedom low rank matrix approximation model selection nuclear norm penalization reduced rank regression Stein's unbiased risk estimator 

摘      要：The objective of this paper is to quantify the complexity of rank and nuclear norm constrained methods for low rank matrix estimation problems. Specifically, we derive analytic forms of the degrees of freedom for these types of estimators in several common settings. These results provide efficient ways of comparing different estimators and eliciting tuning parameters. Moreover, our analyses reveal new insights on the behavior of these low rank matrix estimators. These observations are of great theoretical and practical importance. In particular, they suggest that, contrary to conventional wisdom, for rank constrained estimators the total number of free parameters underestimates the degrees of freedom, whereas for nuclear norm penalization, it overestimates the degrees of freedom. In addition, when using most model selection criteria to choose the tuning parameter for nuclear norm penalization, it oftentimes suffices to entertain a finite number of candidates as opposed to a continuum of choices. Numerical examples are also presented to illustrate the practical implications of our results.