Boosting Unsupervised Monocular Depth Estimation with Auxiliary Semantic Information

作     者：Hui Ren Nan Gao Jia Li Hui Ren;Nan Gao;Jia Li

作者机构：State Key Laboratory of Media Convergence and Communication、Key Laboratory of Acoustic Visual Technology and Intelligent Control SystemMinistry of Culture and Tourism(Communication University of China)、Beijing Key Laboratory of Modern Entertainment Technology(Communication University of China)、School of Information and Communication EngineeringCommunication University of China.No.1Dingfuzhuang StreetChaoyang DistrictBeijing100024China 

出 版 物：《China Communications》 (中国通信（英文版）)

年 卷 期：2021年第18卷第6期

页      面：228-243页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 0809[工学-电子科学与技术（可授工学、理学学位）] 08[工学] 0839[工学-网络空间安全] 080203[工学-机械设计及理论] 0802[工学-机械工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：This work was supported by the national key research development plan(Project No.YS2018YFB1403703) research project of the communication university of china(Project No.CUC200D058). 

主　　题：unsupervised monocular depth estimation semantic segmentation multi-task model 

摘      要：Learning-based multi-task models have been widely used in various scene understanding tasks,and complement each other,i.e.,they allow us to consider prior semantic information to better infer depth.We boost the unsupervised monocular depth estimation using semantic segmentation as an auxiliary task.To address the lack of cross-domain datasets and catastrophic forgetting problems encountered in multi-task training,we utilize existing methodology to obtain redundant segmentation maps to build our cross-domain dataset,which not only provides a new way to conduct multi-task training,but also helps us to evaluate results compared with those of other algorithms.In addition,in order to comprehensively use the extracted features of the two tasks in the early perception stage,we use a strategy of sharing weights in the network to fuse cross-domain features,and introduce a novel multi-task loss function to further smooth the depth values.Extensive experiments on KITTI and Cityscapes datasets show that our method has achieved state-of-the-art performance in the depth estimation task,as well improved semantic segmentation.