Improve Robustness and Accuracy of Deep Neural Network with L_(2,∞) Normalization

作     者：YU Lijia GAO Xiao-Shan YU Lijia;GAO Xiao-Shan

作者机构：Academy of Mathematics and Systems ScienceChinese Academy of SciencesBeijing 100190China University of Chinese Academy of SciencesBeijing 100049China 

出 版 物：《Journal of Systems Science & Complexity》 (系统科学与复杂性学报（英文版）)

年 卷 期：2023年第36卷第1期

页      面：3-28页

核心收录：

学科分类：0710[理学-生物学] 08[工学] 081104[工学-模式识别与智能系统] 0703[理学-化学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：partially supported by NKRDP under Grant No.2018YFA0704705 the National Natural Science Foundation of China under Grant No.12288201. 

主　　题：Deep neural network global robustness measure L_(2,∞)normalization over-fitting Rademacher complexity smooth DNN 

摘      要：In this paper,the L_(2,∞)normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network(DNN)with Relu as activation functions.It is shown that the L_(2,∞)normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions,which reduces over-fitting of the DNN.A global measure is proposed for the robustness of a classification DNN,which is the average radius of the maximal robust spheres with the training samples as centers.A lower bound for the robustness measure in terms of the L_(2,∞)norm is given.Finally,an upper bound for the Rademacher complexity of DNNs with L_(2,∞)normalization is given.An algorithm is given to train DNNs with the L_(2,∞)normalization and numerical experimental results are used to show that the L_(2,∞)normalization is effective in terms of improving the robustness and accuracy.