检索结果-南通市图书馆

Optimization-inspired manual architecture design and neural architecture search

Science China(Information Sciences) 2023年第11期66卷 100-112页

作者： Yibo YANG Zhengyang SHEN Huan LI Zhouchen LIN JD Explore Academy School of Mathematical Sciences Peking University Institute of Robotics and Automatic Information Systems College of Artificial Intelligence Nankai University Key Lab of General Artificial Intelligence School of Intelligence Science and Technology Peking University Institute for Artificial Intelligence Peking University Pazhou Lab

Neural architecture has been a research focus in recent years due to its importance in deciding the performance of deep networks. Representative ones include a residual network(Res Net) with skip connections and a dense network(Dense Net) with dense connections. However, a theoretical guidance for manual architecture design and neural architecture search(NAS) is still lacking. In this paper, we propose a manual architecture design framework, which is inspired by optimization algorithms. It is based on the conjecture that an optimization algorithm with a good convergence rate may imply a neural architecture with good performance. Concretely, we prove under certain conditions that forward propagation in a deep neural network is equivalent to the iterative optimization procedure of the gradient descent algorithm minimizing a cost function. Inspired by this correspondence, we derive neural architectures from fast optimization algorithms,including the heavy ball algorithm and Nesterov’s accelerated gradient descent algorithm. Surprisingly, we find that we can deem the Res Net and Dense Net as special cases of the optimization-inspired *** architectures offer not only theoretical guidance, but also good performances in image recognition on multiple datasets, including CIFAR-10, CIFAR-100, and Image Net. Moreover, we show that our method is also useful for NAS by offering a good initial search point or guiding the search space.

关键词： deep neural network manual architecture design neural architecture search image recognition optimization algorithms learning-based optimization

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：