Semantic segmentation-assisted instance feature fusion for multi-level 3D part instance segmentation

作     者：Chun-Yu Sun Xin Tong Yang Liu Chun-Yu Sun;Xin Tong;Yang Liu

作者机构：Institute for Advanced StudyTsinghua UniversityBeijing 100084China Microsoft Research AsiaBeijing 100080China 

出 版 物：《Computational Visual Media》 (计算可视媒体（英文版）)

年 卷 期：2023年第9卷第4期

页      面：699-715页

核心收录：

学科分类：0710[理学-生物学] 08[工学] 080203[工学-机械设计及理论] 0802[工学-机械工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：3D part instance segmentation feature fusion 3D deep learning 

摘      要：Recognizing 3D part instances from a 3D point cloud is crucial for 3D structure and scene understanding.Several learning-based approaches use semantic segmentation and instance center prediction as training tasks and fail to further exploit the inherent relationship between shape semantics and part instances.In this paper,we present a new method for 3D part instance segmentation.Our method exploits semantic segmentation to fuse nonlocal instance features,such as center prediction,and further enhances the fusion scheme in a multi-and cross-level way.We also propose a semantic region center prediction task to train and leverage the prediction results to improve the clustering of instance points.Our method outperforms existing methods with a large-margin improvement in the PartNet benchmark.We also demonstrate that our feature fusion scheme can be applied to other existing methods to improve their performance in indoor scene instance segmentation tasks.