A novel cross-modal hashing algorithm based on multimodal deep learning

作     者：Wen QU Daling WANG Shi FENG Yifei ZHANG Ge YU 

作者机构：School of Information Science and Engineering Northeastern University Key Laboratory of Medical Image Computing 

出 版 物：《Science China(Information Sciences)》 (中国科学:信息科学(英文版))

年 卷 期：2017年第60卷第9期

页      面：50-63页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 081203[工学-计算机应用技术] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by National Natural Science Foundation of China (Grant Nos. 61402091, 61370074) Fundamental Research Funds for the Central Universities of China (Grant No. N140404012) 

主　　题：hashing cross-modal retrieval cross-modal hashing multimodal data analysis deep learning 

摘      要：With the growing popularity of multimodal data on the Web, cross-modal retrieval on large-scale multimedia databases has become an important research topic. Cross-modal retrieval methods based on hashing assume that there is a latent space shared by multimodal features. To model the relationship among heterogeneous data, most existing methods embed the data into a joint abstraction space by linear projections. However,these approaches are sensitive to noise in the data and are unable to make use of unlabeled data and multimodal data with missing values in real-world applications. To address these challenges, we proposed a novel multimodal deep-learning-based hash(MDLH) algorithm. In particular, MDLH uses a deep neural network to encode heterogeneous features into a compact common representation and learns the hash functions based on the common representation. The parameters of the whole model are fine-tuned in a supervised training *** on two standard datasets show that the method achieves more effective results than other methods in cross-modal retrieval.