检索结果-南通市图书馆

Relative difficulty distillation for semantic segmentation

Science China(Information Sciences) 2024年第9期67卷 126-145页

作者： Dong LIANG Yue SUN Yun DU Songcan CHEN Sheng-Jun HUANG MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Shenzhen Research Institute Nanjing University of Aeronautics and Astronautics

Current knowledge distillation(KD) methods primarily focus on transferring various structured knowledge and designing corresponding optimization goals to encourage the student network to imitate the output of the teacher network. However, introducing too many additional optimization objectives may lead to unstable training, such as gradient conflicts. Moreover, these methods ignored the guidelines of relative learning difficulty between the teacher and student networks. Inspired by human cognitive science, in this paper, we redefine knowledge from a new perspective — the student and teacher networks' relative difficulty of samples, and propose a pixel-level KD paradigm for semantic segmentation named relative difficulty distillation(RDD). We propose a two-stage RDD framework: teacher-full evaluated RDD(TFE-RDD) and teacher-student evaluated RDD(TSE-RDD). RDD allows the teacher network to provide effective guidance on learning focus without additional optimization goals, thus avoiding adjusting learning weights for multiple losses. Extensive experimental evaluations using a general distillation loss function on popular datasets such as Cityscapes, Cam Vid, Pascal VOC, and ADE20k demonstrate the effectiveness of RDD against state-ofthe-art KD methods. Additionally, our research showcases that RDD can integrate with existing KD methods to improve their upper performance bound. Codes are available at https://***/sunyueue/***.

关键词： knowledge distillation semantic segmentation relative difficulty sample weighting prediction discrepancy

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：