检索结果-南通市图书馆

Egocentric Action Anticipation Based on Unsupervised Gaze Estimation

在线全文

学校读者我要写书评

暂无评论

Wuhan University Journal of Natural Sciences 2021年第3期26卷 207-214页

作者： ZHONG Cengsi FANG Zhijun gao yongbin HUANG Bo School of Electronic and Electrical Engineering Shanghai University of Engineering ScienceShanghai 201620China

Gaze information is important for finding region of interest(ROI)which implies where the next action will *** gaze estimation does not work on EPIC-Kitchens for lack of ground *** this paper,we develop an unsupervised gaze estimation method that helps with egocentric action *** adopt gaze map as a feature representation,and input it into a multiple modality network jointly with red-green-blue(RGB),optical flow and object *** explore the method on EGTEA *** estimated gaze map is further optimized with dilation and Gaussian filter,masked onto the original RGB frame and encoded as the important gaze *** results outperform the strong baseline Rolling-Unrolling LSTMs(RULSTM),with top-5 accuracy achieving 34.31%on the seen test set(S1)and 22.07%on unseen test set(S2).The accuracy is improved by 0.58%and 0.87%,respectively.

关键词： egocentric video action anticipation gaze estimation

3D Object Detection Based on Vanishing Point and Prior Orientation

在线全文

学校读者我要写书评

暂无评论

Wuhan University Journal of Natural Sciences 2019年第5期24卷 369-375页

作者： gao yongbin ZHAO Huaqing FANG Zhijun HUANG Bo ZHONG Cengsi Department of Electrical and Electronic Engineering Shanghai University of Engineering Science

3D object detection is one of the most challenging research tasks in computer vision. In order to solve the problem of template information dependency of 3D object proposal in the method of 3D object detection based on 2.5D information, we proposed a 3D object detector based on fusion of vanishing point and prior orientation, which estimates an accurate 3D proposal from 2.5D data, and provides an excellent start point for 3D object classification and localization. The algorithm first calculates three mutually orthogonal vanishing points by the Euler angle principle and projects them into the pixel coordinate system. Then, the top edge of the 2D proposal is sampled by the preset sampling pitch, and the first one vertex is taken. Finally, the remaining seven vertices of the 3D proposal are calculated according to the linear relationship between the three vanishing points and the vertices, and the complete information of the 3D proposal is obtained. The experimental results show that this proposed method improves the Mean Average Precision score by 2.7% based on the Amodal3Det method.

关键词： image analysis 3D object detection prior orientation vanishing point Euler angle

Knowledge Enhanced Pre-Training Model for Vision-Language-Navigation Task

在线全文

学校读者我要写书评

暂无评论

Wuhan University Journal of Natural Sciences 2021年第2期26卷 147-155页

作者： HUANG Jitao ZENG Guohui HUANG Bo gao yongbin LIU Jin SHI Zhicai College of Electrical and Electronic Engineering Shanghai University of Engineering ScienceShanghai 201620China Shanghai Key Laboratory of Integrated Administration Technologies for Information Security Shanghai 200240China

Vision-Language-Navigation(VLN) task is a cross-modality task that combines natural language processing and computer vision. This task requires the agent to automatically move to the destination according to the natural language instruction and the observed surrounding visual information. To make the best decision, in every step during the navigation, the agent should pay more attention to understanding the objects, the object attributes, and the object relationships. But most current methods process all received textual and visual information equally. Therefore, this paper integrates more detailed semantic connections between visual and textual information through three pre-training tasks(object prediction, object attributes prediction, and object relationship prediction). The model will learn better fusion representation and alignment between these two types of information to improve the success rate(SR) and generalization. The experiments show that compared with the former baseline models, the SR on the unseen validation set(Val Unseen) increased by 7%, and the SR weighted by path length(SPL) increased by 7%;the SR on the test set(Test) increased 4%, SPL increased by 3%.

关键词： pre-training cross-modality deep learning scene graph

3D Object Detection Incorporating Instance Segmentation and Image Restoration

在线全文

学校读者我要写书评

暂无评论

Wuhan University Journal of Natural Sciences 2019年第4期24卷 360-368页

作者： HUANG Bo HUANG Man gao yongbin YU Yuxin JIANG Xiaoyan ZHANG Juan Department of Electrical and Electronic Engineering Shanghai University of Engineering Science Shanghai 201620 China Collaborative Innovation Center for Economics Crime Investigation and Prevention Technology Nanchang 330103 Jiangxi China School of Economics and Finance Shanghai International Studies University Shanghai 210620 China

Nowadays, 3D object detection, which uses the color and depth information to find object localization in the 3D world and estimate their physical size and pose, is one of the most important 3D perception tasks in the field of computer vision. In order to solve the problem of mixed segmentation results when multiple instances appear in one frustum in the F-PointNet method and in the occlusion that leads to the loss of depth information, a 3D object detection approach based on instance segmentation and image restoration is proposed in this paper. Firstly, instance segmentation with Mask R-CNN on an RGB image is used to avoid mixed segmentation results. Secondly, for the detected occluded objects, we remove the occluding object first in the depth map and then restore the empty pixel region by utilizing the Criminisi Algorithm to recover the missing depth information of the object. The experimental results show that the proposed method improves the average precision score compared with the F-PointNet method.

关键词： image processing 3D object detection instance segmentation depth information image restoration

Keypoints and Descriptors Based on Cross-Modality Information Fusion for Camera Localization

在线全文

学校读者我要写书评

暂无评论

Wuhan University Journal of Natural Sciences 2021年第2期26卷 128-136页

作者： MA Shuo gao yongbin+ TIAN Fangzheng LU Junxin HUANG Bo GU Jia ZHOU Yilong College of Electronic and Electrical Engineering Shanghai University of Engineering ScienceShanghai 201620China

To address the problem that traditional keypoint detection methods are susceptible to complex backgrounds and local similarity of images resulting in inaccurate descriptor matching and bias in visual localization, keypoints and descriptors based on cross-modality fusion are proposed and applied to the study of camera motion estimation. A convolutional neural network is used to detect the positions of keypoints and generate the corresponding descriptors, and the pyramid convolution is used to extract multi-scale features in the network. The problem of local similarity of images is solved by capturing local and global feature information and fusing the geometric position information of keypoints to generate descriptors. According to our experiments, the repeatability of our method is improved by 3.7%, and the homography estimation is improved by 1.6%. To demonstrate the practicability of the method, the visual odometry part of simultaneous localization and mapping is constructed and our method is 35% higher positioning accuracy than the traditional method.

关键词： keypoints descriptors cross-modality information global feature visual odometry

基于非局部卷积和卷积注意力模块的眩晕眼震诊断方法

在线全文

学校读者我要写书评

暂无评论

中国医学物理学杂志 2024年第5期41卷 571-578页

作者：贺斌高永彬上海工程技术大学电子电气工程学院上海201620

鉴于良性阵发性位置性眩晕的复杂致病因素和诊断困难等问题,提出一种新的基于非局部卷积和卷积注意力模块(CBAM)的眩晕眼震诊断方法。首先,通过构建目标检测模型定位瞳孔,从而捕捉眼球运动并提取水平和垂直运动轨迹时序数据。其次,采用... 详细信息

鉴于良性阵发性位置性眩晕的复杂致病因素和诊断困难等问题,提出一种新的基于非局部卷积和卷积注意力模块(CBAM)的眩晕眼震诊断方法。首先,通过构建目标检测模型定位瞳孔,从而捕捉眼球运动并提取水平和垂直运动轨迹时序数据。其次,采用分类模型进行分类检测,该分类模型采用非局部卷积来捕获眼震数据中的远程依赖关系特征,并引入CBAM来提取特征层中的高级和低级语义信息,从而提高了分类模型的检测性能。在眼耳鼻喉科医院提供的视频眼震数据集上进行了实验,结果表明,与主流方法相比,本文所提出的诊断方法在精确率、召回率、准确率、平均F1值等评估指标上比主流方法分别提高了1.82%、2.09%、1.62%和1.96%,表明了本文方法的显著性优势。

关键词：良性阵发性位置性眩晕医学图像处理时序数据分类目标检测视频眼震数据分类

维普期刊数据库博看期刊

基于嵌入注意力机制的X光脊椎骨角点定位模型

在线全文

学校读者我要写书评

暂无评论

激光与光电子学进展 2023年第10期60卷 242-249页

作者：陈瑶余文俊高永彬上海工程技术大学电子电气工程学院上海201620

脊柱侧弯是当今社会中常见的脊柱疾病,在X光图像上快速而准确地定位脊椎骨角点并计算其Cobb角度数是医生诊断脊柱弯曲程度的金指标。针对X光骨科图片中其他器官的遮挡以及复杂背景干扰等问题,提出一种基于嵌入注意力机制和向量损失模块... 详细信息

脊柱侧弯是当今社会中常见的脊柱疾病,在X光图像上快速而准确地定位脊椎骨角点并计算其Cobb角度数是医生诊断脊柱弯曲程度的金指标。针对X光骨科图片中其他器官的遮挡以及复杂背景干扰等问题,提出一种基于嵌入注意力机制和向量损失模块的神经网络模型。所提模型以vertebra-focused landmark detection(VFLD)网络为基础网络,在编码器和解码器之间嵌入旋转注意力机制模块加强网络对于脊椎骨深层、高维特征的提取,抑制其他器官的干扰,同时利用向量相似性的损失函数对网络进行训练。实验结果表明,在MICCAI 2019公开脊椎挑战赛数据集中,所提模型的对称平均绝对百分比误差准确度高达9.31,可以有效提高原模型检测椎骨角点能力。与现有的诸多模型相比,其具有较高的准确率和稳健性。

关键词：图像处理 X光图像神经网络注意力机制 Cobb角

同方期刊数据库

嵌入注意力机制的多模型融合冠脉CTA分割算法

在线全文

学校读者我要写书评

暂无评论

计算机科学与探索 2020年第9期14卷 1602-1611页

作者：沈烨方志军高永彬上海工程技术大学电子电气工程学院

冠心病是当今世界上最大的健康问题之一,因此对冠心病的早期预防和诊断非常重要。斑块的存在和冠状动脉血管狭窄是引发冠心病的主要原因,对斑块的检测和冠状动脉血管分割成为检测冠状动脉疾病的首选方案。目前手动分割冠状动脉耗时并且... 详细信息

冠心病是当今世界上最大的健康问题之一,因此对冠心病的早期预防和诊断非常重要。斑块的存在和冠状动脉血管狭窄是引发冠心病的主要原因,对斑块的检测和冠状动脉血管分割成为检测冠状动脉疾病的首选方案。目前手动分割冠状动脉耗时并且由操作者的主观意识决定,这使得现在的临床医学诊断中对自动分割技术的需要显而易见。提出了一种基于深度学习多模型融合的冠脉CT血管造影(CTA)的血管分割方法,该方法包括三个网络模型:一个原始三维全卷积网络(3D FCN),以及两个在原始3D FCN中嵌入注意力门控(AG)模型的网络。然后把三种网络的预测结果采用多数投票算法进行融合来得到网络预测的最终结果。在后处理阶段,采用水平集函数对网络融合预测的结果进行进一步迭代优化。在评估所提出方法的过程中把JI和DSC分数来作为性能度量进行比较,最终结果表明,所提出的方法提供了更好更准确的分割结果。

关键词：冠脉CTA 多模型融合三维全卷积网络(3D FCN) 注意力门控模型水平集函数

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

结合混合域注意力与空洞卷积的3维目标检测

中国图象图形学报 2020年第6期25卷 1221-1234页

作者：严娟方志军高永彬上海工程技术大学电子电气工程学院上海201620

目的通过深度学习卷积神经网络进行3维目标检测的方法已取得巨大进展,但卷积神经网络提取的特征既缺乏不同区域特征的依赖关系,也缺乏不同通道特征的依赖关系,同时难以保证在无损空间分辨率的情况下扩大感受野。针对以上不足,提出了一... 详细信息

目的通过深度学习卷积神经网络进行3维目标检测的方法已取得巨大进展,但卷积神经网络提取的特征既缺乏不同区域特征的依赖关系,也缺乏不同通道特征的依赖关系,同时难以保证在无损空间分辨率的情况下扩大感受野。针对以上不足,提出了一种结合混合域注意力与空洞卷积的3维目标检测方法。方法在输入层融入空间域注意力机制,变换输入信息的空间位置,保留需重点关注的区域特征;在网络中融入通道域注意力机制,提取特征的通道权重,获取关键通道特征;通过融合空间域与通道域注意力机制,对特征进行混合空间与通道的混合注意。在特征提取器的输出层融入结合空洞卷积与通道注意力机制的网络层,在不损失空间分辨率的情况下扩大感受野,根据不同感受野提取特征的通道权重后进行融合,得到全局感受野的关键通道特征;引入特征金字塔结构构建特征提取器,提取高分辨率的特征图,大幅提升网络的检测性能。运用基于二阶段的区域生成网络,回归定位更准确的3维目标框。结果KITTI(A project of Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago)数据集中的实验结果表明,在物体被遮挡的程度由轻到高时,对测试集中的car类别,3维目标检测框的平均精度AP3D值分别为83.45%、74.29%、67.92%,鸟瞰视角2维目标检测框的平均精度APBEV值分别为89.61%、87.05%、79.69%;对pedestrian和cyclist类别,AP3D和APBEV值同样比其他方法的检测结果有一定优势。结论本文提出的3维目标检测网络,一定程度上解决了3维检测任务中卷积神经网络提取的特征缺乏视觉注意力的问题,从而使3维目标检测更有效地运用于室外自动驾驶。

关键词： 3维目标检测注意力机制空洞卷积感受野金字塔网络卷积神经网络(CNN)