Enhancing Image Description Generation through Deep Reinforcement Learning:Fusing Multiple Visual Features and Reward Mechanisms

作     者：Yan Li Qiyuan Wang Kaidi Jia 

作者机构：School of Cyber SecurityGansu University of Political Science and LawLanzhou730070China 

出 版 物：《Computers, Materials & Continua》 (计算机、材料和连续体（英文）)

年 卷 期：2024年第78卷第2期

页      面：2469-2489页

核心收录：

学科分类：08[工学] 0805[工学-材料科学与工程（可授工学、理学学位）] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：This research was funded by the Natural Science Foundation of Gansu Province with Approval Numbers 20JR10RA334 and 21JR7RA570 Funding is provided for the 2021 Longyuan Youth Innovation and Entrepreneurship Talent Project with Approval Number 2021LQGR20 the University Level Innovation Project with Approval NumbersGZF2020XZD18 jbzxyb2018-01 of Gansu University of Political Science and Law 

主　　题：Image description deep reinforcement learning attention mechanism 

摘      要：Image description task is the intersection of computer vision and natural language processing,and it has important prospects,including helping computers understand images and obtaining information for the visually *** study presents an innovative approach employing deep reinforcement learning to enhance the accuracy of natural language descriptions of *** method focuses on refining the reward function in deep reinforcement learning,facilitating the generation of precise descriptions by aligning visual and textual features more *** approach comprises three key ***,it utilizes Residual Network 101(ResNet-101)and Faster Region-based Convolutional Neural Network(Faster R-CNN)to extract average and local image features,respectively,followed by the implementation of a dual attention mechanism for intricate feature ***,the Transformer model is engaged to derive contextual semantic features from textual ***,the generation of descriptive text is executed through a two-layer long short-term memory network(LSTM),directed by the value and reward *** with the image description method that relies on deep learning,the score of Bilingual Evaluation Understudy(BLEU-1)is 0.762,which is 1.6%higher,and the score of BLEU-4 is ***-based Image Description Evaluation(CIDEr)scored 0.998,Recall-Oriented Understudy for Gisting Evaluation(ROUGE)scored 0.552,the latter improved by 0.36%.These results not only attest to the viability of our approach but also highlight its superiority in the realm of image *** research can explore the integration of our method with other artificial intelligence(AI)domains,such as emotional AI,to create more nuanced and context-aware systems.