检索结果-南通市图书馆

On the learning dynamics of two-layer quadratic neural networks for understanding deep learning

Frontiers of Computer Science 2022年第3期16卷 77-82页

作者： Zhenghao TAN Songcan CHEN College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing 211106China College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsMIIT Key Laboratory of Pattern Analysis and Machine IntelligenceNanjing 211106China

Deep learning performs as a powerful paradigm in many real-world applications;however,its mechanism remains much of a *** gain insights about nonlinear hierarchical deep networks,we theoretically describe the coupled nonlinear learning dynamic of the two-layer neural network with quadratic activations,extending existing results from the linear *** quadratic activation,although rarely used in practice,shares convexity with the widely used ReLU activation,thus producing similar *** this work,we focus on the case of a canonical regression problem under the standard normal distribution and use a coupled dynamical system to mimic the gradient descent method in the sense of a continuous-time limit,then use the high order moment tensor of the normal distribution to simplify these ordinary differential *** simplified system yields unexpected fixed *** existence of these non-global-optimal stable points leads to the existence of saddle points in the loss surface of the quadratic *** analysis shows there are conserved quantities during the training of the quadratic *** quantities might result in a failed learning process if the network is initialized ***,We illustrate the comparison between the numerical learning curves and the theoretical one,which reveals the two alternately appearing stages of the learning process.

关键词： learning dynamic quadratic network ordinary differential equations

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：