检索结果-南通市图书馆

Vision Transformers with Hierarchical Attention

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2024年第4期21卷 670-683页

作者： Yun Liu Yu-Huan Wu Guolei Sun Le Zhang Ajad Chhatkuli luc van gool Institute for Infocomm Research(12R) A*STARSingapore 138632Singapore Institute of High Performance Computing(IHPC) A*STARSingapore 138632Singapore Computer Vision Lab ETH ZurichZurich 8092Switzerland School of Information and Communication Engineering University of Electronic Science and Technology of China(UESTC)Chengdu 611731China

This paper tackles the high computational/space complexity associated with multi-head self-attention(MHSA)in vanilla vision *** this end,we propose hierarchical MHSA(H-MHSA),a novel approach that computes self-attention in a hierarchical ***,we first divide the input image into patches as commonly done,and each patch is viewed as a ***,the proposed H-MHSA learns token relationships within local patches,serving as local relationship ***,the small patches are merged into larger ones,and H-MHSA models the global dependencies for the small number of the merged *** last,the local and global attentive features are aggregated to obtain features with powerful representation *** we only calculate attention for a limited number of tokens at each step,the computational load is reduced ***,H-MHSA can efficiently model global relationships among tokens without sacrificing fine-grained *** the H-MHSA module incorporated,we build a family of hierarchical-attention-based transformer networks,namely *** demonstrate the superiority of HAT-Net in scene understanding,we conduct extensive experiments on fundamental vision tasks,including image classification,semantic segmentation,object detection and instance ***,HAT-Net provides a new perspective for vision *** and pretrained models are available at https://***/yun-liu/HAT-Net.

关键词： Vision transformer hierarchical attention global attention local attention scene understanding.

Rethinking Global Context in Crowd Counting

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2024年第4期21卷 640-651页

作者： Guolei Sun Yun Liu Thomas Probst Danda Pani Paudel Nikola Popovic luc van gool Computer Vision Lab ETH ZurichZurich 8092Switzerland Institute for Infocomm Research A*STARSingapore 138632Singapore Magic Leap Zirich 8050Switzerland

This paper investigates the role of global context for crowd ***,a pure transformer is used to extract features with global information from overlapping image *** by classification,we add a context token to the input sequence,to facilitate information exchange with tokens corresponding to image patches throughout transformer *** to the fact that transformers do not explicitly model the tried-and-true channel-wise interactions,we propose a token-attention module(TAM)to recalibrate encoded features through channel-wise attention informed by the context *** that,it is adopted to predict the total person count of the image through regression-token module(RTM).Extensive experiments on various datasets,including ShanghaiTech,UCFQNRF,JHU-CROWD++and NWPU,demonstrate that the proposed context extraction techniques can significantly improve the performanceover the baselines.

关键词： Crowd counting vision transformer global context attention density map.

Facial-sketch Synthesis:A New Challenge

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2022年第4期19卷 257-287页

作者： Deng-Ping Fan Ziling Huang Peng Zheng Hong Liu Xuebin Qin luc van gool Computer Vision Laboratory ETH ZürichZürich 8092Switzerland Information and Communication Engineering University of TokyoTokyo 113-8654Japan Computer Vision Mohamed bin Zayed University of Artificial IntelligenceAbu DhabiUAE Digital Content and Media Sciences Research Division National Institute of InformaticsTokyo 101-8430Japan

This paper aims to conduct a comprehensive study on facial-sketch synthesis(FSS).However,due to the high cost of obtaining hand-drawn sketch datasets,there is a lack of a complete benchmark for assessing the development of FSS algorithms over the last *** first introduce a high-quality dataset for FSS,named FS2K,which consists of 2104 image-sketch pairs spanning three types of sketch styles,image backgrounds,lighting conditions,skin colors,and facial ***2K differs from previous FSS datasets in difficulty,diversity,and scalability and should thus facilitate the progress of FSS ***,we present the largest-scale FSS investigation by reviewing 89 classic methods,including 25 handcrafted feature-based facial-sketch synthesis approaches,29 general translation methods,and 35 image-to-sketch *** addition,we elaborate comprehensive experiments on the existing 19 cutting-edge ***,we present a simple baseline for FSS,named *** only two straightforward components,i.e.,facialaware masking and style-vector expansion,our FSGAN surpasses the performance of all previous state-of-the-art models on the proposed FS2K dataset by a large ***,we conclude with lessons learned over the past years and point out several unsolved *** code is available at https://***/DengPingFan/FSGAN.

关键词： Facial sketch synthesis(FSS) facial sketch dataset benchmark attribute style transfer

维普期刊数据库

Masked Vision-language Transformer in Fashion

在线全文

学校读者我要写书评

暂无评论

Machine Intelligence Research 2023年第3期20卷 421-434页

作者： Ge-Peng Ji Mingchen Zhuge Dehong Gao Deng-Ping Fan Christos Sakaridis luc van gool International Core Business Unit Alibaba GroupHangzhou 310051China Computer Vision Lab ETH ZürichZürich 8092Switzerland

We present a masked vision-language transformer(MVLT)for fashion-specific multi-modal ***,we simply utilize the vision transformer architecture for replacing the bidirectional encoder representations from Transformers(BERT)in the pre-training model,making MVLT the first end-to-end framework for the fashion ***,we designed masked image reconstruction(MIR)for a fine-grained understanding of *** is an extensible and convenient architecture that admits raw multimodal inputs without extra pre-processing models(e.g.,ResNet),implicitly modeling the vision-language *** importantly,MVLT can easily generalize to various matching and generative *** results show obvious improvements in retrieval(rank@5:17%)and recognition(accuracy:3%)tasks over the Fashion-Gen 2018 winner,*** code is available at https://***/GewelsJI/MVLT.

关键词： Vision-language masked image reconstruction transformer fashion e-commercial

SAM struggles in concealed scenes—empirical study on “Segment Anything”

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2023年第12期66卷 278-280页

作者： Ge-Peng JI Deng-Ping FAN Peng XU Bowen ZHOU Ming-Ming CHENG luc van gool School of Computing Australian National University Computer Vision Lab(CVL) ETH Zurich Department of Electronic Engineering Tsinghua University College of Computer Science Nankai University

Large models open up new opportunities for artificial intelligence. In the past few months, there has been a boom in training foundation models on the vast linguistic corpus to produce amazing applications, e.g., Chat GPT, *** natural language processing and multimodal learning communities have been revolutionized. Large models’ capacity for generalization and emergent makes it easy for users to believe that large models can solve anything.

关键词：

Deep Gradient Learning for Efficient Camouflaged Object Detection

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2023年第1期20卷 92-108页

作者： Ge-Peng Ji Deng-Ping Fan Yu-Cheng Chou Dengxin Dai Alexander Liniger luc van gool School of Computer Science Wuhan UniversityWuhan 430072China Computer Vision Laboratory ETH ZürichZürich 8092Switzerland

This paper introduces deep gradient network(DGNet),a novel deep framework that exploits object gradient supervision for camouflaged object detection(COD).It decouples the task into two connected branches,i.e.,a context and a texture *** es-sential connection is the gradient-induced transition,representing a soft grouping between context and texture *** from the simple but efficient framework,DGNet outperforms existing state-of-the-art COD models by a large ***,our efficient version,DGNet-S,runs in real-time(80 fps)and achieves comparable results to the cutting-edge model JCSOD-CVPR21 with only 6.82%*** application results also show that the proposed DGNet performs well in the polyp segmentation,defect detec-tion,and transparent object segmentation *** code will be made available at https://***/GewelsJI/DGNet.

关键词： Camouflaged object detection(COD) object gradient soft grouping efficient model image segmentation

维普期刊数据库

How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges

在线全文

学校读者我要写书评

暂无评论

Machine Intelligence Research 2023年第5期20卷 605-613页

作者： Haotong Qin Ge-Peng Ji Salman Khan Deng-Ping Fan Fahad Shahbaz Khan luc van gool Computer Vision Lab(CVL) ETH ZurichZurich 8001Switzerland College of Engineering Computing&CyberneticsAustralian National UniversityCanberra 8105Australia Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi 999041UAE

Google's Bard has emerged as a formidable competitor to OpenAI's ChatGPT in the field of conversational ***,Bard has recently been updated to handle visual inputs alongside text prompts during *** Bard's impressive track record in handling textual inputs,we explore its capabilities in understanding and interpreting visual data(images)conditioned by text *** exploration holds the potential to unveil new insights and challenges for Bard and other forthcoming multi-modal Generative models,especially in addressing complex computer vision problems that demand accurate visual and language ***,in this study,we focus on 15 diverse task scenarios encompassing regular,camouflaged,medical,under-water and remote sensing data to comprehensively evaluate Bard's *** primary finding indicates that Bard still struggles in these vision scenarios,highlighting the significant gap in vision-based understanding that needs to be bridged in future *** expect that this empirical study will prove valuable in advancing future models,leading to enhanced capabilities in comprehending and interpreting finegrained visual *** project is released on https://***/htqin/GoogleBard-VisUnderstand.

关键词： Google Bard multi-modal understanding visual comprehension large language models conversational AI chatbot.

Video Polyp Segmentation: A Deep Learning Perspective

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2022年第6期19卷 531-549页

作者： Ge-Peng Ji Guobao Xiao Yu-Cheng Chou Deng-Ping Fan Kai Zhao Geng Chen luc van gool Research School of Engineering Australian National UniversityCanberra 2601Australia College of Computer and Control Engineering Minjiang UniversityFuzhou 350108China Department of Computer Science Johns Hopkins UniversityBaltimore 21218USA Computer Vision Laboratory ETH ZürichZürich 8092Switzerland Department of Radiological Sciences University of CaliforniaLos Angeles 90095USA School of Computer Science and Engineering Northwestern Polytechnical UniversityXi′an 710072China

We present the first comprehensive video polyp segmentation(VPS)study in the deep learning *** the years,developments in VPS are not moving forward with ease due to the lack of a large-scale dataset with fine-grained segmentation *** address this issue,we first introduce a high-quality frame-by-frame annotated VPS dataset,named SUN-SEG,which contains 158690colonoscopy video frames from the well-known *** provide additional annotation covering diverse types,i.e.,attribute,object mask,boundary,scribble,and ***,we design a simple but efficient baseline,named PNS+,which consists of a global encoder,a local encoder,and normalized self-attention(NS)*** global and local encoders receive an anchor frame and multiple successive frames to extract long-term and short-term spatial-temporal representations,which are then progressively refined by two NS *** experiments show that PNS+achieves the best performance and real-time inference speed(170 fps),making it a promising solution for the VPS ***,we extensively evaluate 13 representative polyp/object segmentation models on our SUN-SEG dataset and provide attribute-based ***,we discuss several open issues and suggest possible research directions for the VPS *** project and dataset are publicly available at https://***/GewelsJI/VPS.

关键词： Video polyp segmentation(VPS) dataset self-attention colonoscopy abdomen

维普期刊数据库

Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis

在线全文

学校读者我要写书评

暂无评论

Machine Intelligence Research 2023年第6期20卷 822-836页

作者： Kai Zhang Yawei Li Jingyun Liang Jiezhang Cao Yulun Zhang Hao Tang Deng-Ping Fan Radu Timofte luc van gool Computer Vision Lab ETH ZürichZürich8092Switzerland Computer Vision Lab University of WürzburgWürzburg97074Germany KU Leuven Leuven3000Belgium

While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising,existing methods mostly rely on simple noise assumptions,such as additive white Gaussian noise(AWGN),JPEG compression noise and camera sensor noise,and a general-purpose blind denoising method for real images remains *** this paper,we attempt to solve this problem from the perspective of network architecture design and training data ***,for the network architecture design,we propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block,and then plug it as the main building block into the widely-used image-to-image translation UNet *** the training data synthesis,we design a practical noise degradation model which takes into consideration different kinds of noise(including Gaussian,Poisson,speckle,JPEG compression,and processed camera sensor noises)and resizing,and also involves a random shuffle strategy and a double degradation *** experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance and the new degradation model can help to significantly improve the *** believe our work can provide useful insights into current denoising *** source code is available at https://***/cszn/SCUNet.

关键词： Blind image denoising real image denosing data synthesis Transformer image signal processing(ISP)pipeline