Keypoints and Descriptors Based on Cross-Modality Information Fusion for Camera Localization

作     者：MA Shuo GAO Yongbin+ TIAN Fangzheng LU Junxin HUANG Bo GU Jia ZHOU Yilong MA Shuo;GAO Yongbin;TIAN Fangzheng;LU Junxin;HUANG Bo;GU Jia;ZHOU Yilong

作者机构：College of Electronic and Electrical EngineeringShanghai University of Engineering ScienceShanghai 201620China 

出 版 物：《Wuhan University Journal of Natural Sciences》 (武汉大学学报（自然科学英文版）)

年 卷 期：2021年第26卷第2期

页      面：128-136页

核心收录：

学科分类：081203[工学-计算机应用技术] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Supported by the National Natural Science Foundation of China (61802253) 

主　　题：keypoints descriptors cross-modality information global feature visual odometry 

摘      要：To address the problem that traditional keypoint detection methods are susceptible to complex backgrounds and local similarity of images resulting in inaccurate descriptor matching and bias in visual localization, keypoints and descriptors based on cross-modality fusion are proposed and applied to the study of camera motion estimation. A convolutional neural network is used to detect the positions of keypoints and generate the corresponding descriptors, and the pyramid convolution is used to extract multi-scale features in the network. The problem of local similarity of images is solved by capturing local and global feature information and fusing the geometric position information of keypoints to generate descriptors. According to our experiments, the repeatability of our method is improved by 3.7%, and the homography estimation is improved by 1.6%. To demonstrate the practicability of the method, the visual odometry part of simultaneous localization and mapping is constructed and our method is 35% higher positioning accuracy than the traditional method.