Fusing Geometrical and Visual Information via Superpoints for the Semantic Segmentation of 3D Road Scenes

作     者：Liuyuan Deng Ming Yang Zhidong Liang Yuesheng He Chunxiang Wang Liuyuan Deng;Ming Yang;Zhidong Liang;Yuesheng He;Chunxiang Wang

作者机构：the Department of AutomationShanghai Jiao Tong UniversityShanghai 200240China the Key Laboratory of System Control and Information ProcessingMinistry of Education of ChinaShanghai 200240China the Research Institute of RoboticsShanghai Jiao Tong UniversityShanghai 200240China 

出 版 物：《Tsinghua Science and Technology》 (清华大学学报（自然科学版（英文版）)

年 卷 期：2020年第25卷第4期

页      面：498-507页

核心收录：

学科分类：08[工学] 0838[工学-公安技术] 

基　　金：supported by the National Natural Science Foundation of China(No.U1764264/61873165) Shanghai Automotive Industry Science and Technology Development Foundation(No.1807) the International Chair on Automated Driving of Ground Vehicle 

主　　题：scene understanding point cloud semantic segmentation multi-modal information fusion deep learning 

摘      要：This paper addresses the problem of the semantic segmentation of large-scale 3D road scenes by incorporating the complementary advantages of point clouds and *** make full use of geometrical and visual information,this paper extracts 3D geometric features from a point cloud using a deep neural network for 3D semantic segmentation and extracts 2D visual features from images using a Convolutional Neural Network(CNN)for 2D semantic *** order to bridge the features of the two modalities,this paper uses superpoints as an intermediate representation to connect the 2D features with the 3D features.A superpoint-based pooling method is proposed to fuse the features from the two different modalities for joint *** evaluate the approach,the paper generates 3D scenes from the Virtual KITTI *** results of the experiments demonstrate that the proposed approach is capable of segmenting large-scale 3D road scenes based on the compact and semantically homogeneous superpoints,and that it achieves considerable improvements over the 2D image and 3D point cloud semantic segmentation methods.