Transformers in medical image analysis

作     者：Kelei He Chen Gan Zhuoyuan Li Islem Rekik Zihao Yin Wen Ji Yang Gao Qian Wang Junfeng Zhang Dinggang Shen Kelei He;Chen Gan;Zhuoyuan Li;Islem Rekik;Zihao Yin;Wen Ji;Yang Gao;Qian Wang;Junfeng Zhang;Dinggang Shen

作者机构：Medical School of Nanjing UniversityNanjingJiangsu 210093China National Institute of Healthcare Data Science at Nanjing UniversityNanjingJiangsu 210093China BASIRA LaboratoryFaculty of Computer and Informatics EngineeringIstanbul Technical UniversityIstanbulTurkey School of Science and EngineeringComputingUniversity of DundeeUK State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingJiangsu 210093China School of Biomedical EngineeringShanghaiTech UniversityShanghai 201210China Department of Research and DevelopmentShanghai United Imaging Intelligence Co.Ltd.Shanghai 200030China Shanghai Clinical Research and Trial CenterShanghai 201703China 

出 版 物：《Intelligent Medicine》 (智慧医学（英文）)

年 卷 期：2023年第3卷第1期

页      面：59-78页

核心收录：

学科分类：1001[医学-基础医学(可授医学、理学学位)] 08[工学] 080203[工学-机械设计及理论] 0802[工学-机械工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：the National Natural Science Foundation of China(Grant No.62106101) the Natural Science Foundation of Jiangsu Province(Grant No.BK20210180). 

主　　题：Transformer Medical image analysis Deep learning Diagnosis Registration Segmentation Image synthesis Multi-task learning Multi-modal learning Weakly-supervised learning 

摘      要：Transformers have dominated the field of natural language processing and have recently made an impact in the area of computer vision.In the field of medical image analysis,transformers have also been successfully used in to full-stack clinical applications,including image synthesis/reconstruction,registration,segmentation,detection,and diagnosis.This paper aimed to promote awareness of the applications of transformers in medical image analysis.Specifically,we first provided an overview of the core concepts of the attention mechanism built into transformers and other basic components.Second,we reviewed various transformer architectures tailored for medical image applications and discuss their limitations.Within this review,we investigated key challenges including the use of transformers in different learning paradigms,improving model efficiency,and coupling with other techniques.We hope this review would provide a comprehensive picture of transformers to readers with an interest in medical image analysis.