Understanding and Generating Ultrasound Image Description

作     者：Xian-Huu Zeng Bang-Gui Liu Meng Zhou 

作者机构：School of Computer Science and Technology Chongqing University of Posts and Telecommunications Chongqing 400065 China 

出 版 物：《Journal of Computer Science & Technology》 (计算机科学技术学报（英文版）)

年 卷 期：2018年第33卷第5期

页      面：1086-1100页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 080103[工学-流体力学] 080203[工学-机械设计及理论] 080104[工学-工程力学] 081502[工学-水力学及河流动力学] 0815[工学-水利工程] 0835[工学-软件工程] 0802[工学-机械工程] 0701[理学-数学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 0801[工学-力学（可授工学、理学学位）] 

基　　金：国家自然科学基金 the Chongqing Natural Science Foundation Program of China Postgraduate Scientific Research and Innovation Project of Chongqing of China 

主　　题：ultrasound image fine-grained classification image captioning 

摘      要：To understand the content of ultrasound images more conveniently and more quickly, in this paper, we propose a coarse-to-fine ultrasound image captioning ensemble model, which can automatically generate the annotation text that is composed of relevant n-grams to describe the disease information in the ultrasound images. First, the organs in the ultrasound images are detected by the coarse classification model. Second, the ultrasound images are encoded by the corresponding fine-grained classification model according to the organ labels. Finally, we input the encoding vectors to the language generation model, and the language generation model generates automatically annotation text to describe the disease information in the ultrasound images. In our experiments, the encoding model can obtain the high accuracy rate in the ultrasound image recognition. And the language generation model can automatically generate high-quality annotation text. In practical applications, the coarse-to-fine ultrasound image captioning ensemble model can help patients and doctors obtain the well understanding of the contents of ultrasound images.