检索结果-南通市图书馆

Alignment efficient image-sentence retrieval considering transferable cross-modal representation learning

Frontiers of Computer Science 2024年第1期18卷 77-91页

作者： Yang YANG Jinyi GUO Guangyu LI Lanyu LI Wenjie LI Jian YANG School of Computer Science and Engineering Nanjing University of Science and TechnologyNanjing 210094China Department of Computing Hong Kong Polytechnic UniversityHong Kong 100872China State Key Lab.for Novel Software Technology Nanjing UniversityNanjing 210094China 14th Research Institute of China Electronics Technology Group Corporation Nanjing 210094China guangyu.li2017@***

Traditional image-sentence cross-modal retrieval methods usually aim to learn consistent representations of heterogeneous modalities,thereby to search similar instances in one modality according to the query from another modality in *** basic assumption behind these methods is that parallel multi-modal data(i.e.,different modalities of the same example are aligned)can be obtained in *** other words,the image-sentence cross-modal retrieval task is a supervised task with the alignments as ***,in many real-world applications,it is difficult to realign a large amount of parallel data for new scenarios due to the substantial labor costs,leading the non-parallel multi-modal data and existing methods cannot be used *** the other hand,there actually exists auxiliary parallel multi-modal data with similar semantics,which can assist the non-parallel data to learn the consistent ***,in this paper,we aim at“Alignment Efficient Image-Sentence Retrieval”(AEIR),which recurs to the auxiliary parallel image-sentence data as the source domain data,and takes the non-parallel data as the target domain *** single-modal transfer learning,AEIR learns consistent image-sentence cross-modal representations of target domain by transferring the alignments of existing parallel ***,AEIR learns the image-sentence consistent representations in source domain with parallel data,while transferring the alignment knowledge across domains by jointly optimizing a novel designed cross-domain cross-modal metric learning based constraint with intra-modal domain adversarial ***,we can effectively learn the consistent representations for target domain considering both the structure and semantic ***,extensive experiments on different transfer scenarios validate that AEIR can achieve better retrieval results comparing with the baselines.

关键词： image-sentence retrieval transfer learning semantic transfer structure transfer

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：