Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

作     者：Chunming Wu Wukai Liu Xin Ma 

作者机构：Key Laboratory of Modern Power System Simulation and Control&Renewable Energy TechnologySchool of Electrical EngineeringNortheast Electric Power UniversityJilin132012China School of Electrical EngineeringNortheast Electric Power UniversityJilin132012China School of Aeronautical EngineeringJilin Institute of Chemical TechnologyJilin132022China 

出 版 物：《Computers, Materials & Continua》 (计算机、材料和连续体（英文）)

年 卷 期：2024年第79卷第4期

页      面：1441-1461页

核心收录：

学科分类：080901[工学-物理电子学] 0809[工学-电子科学与技术（可授工学、理学学位）] 08[工学] 080401[工学-精密仪器及机械] 080203[工学-机械设计及理论] 0804[工学-仪器科学与技术] 0805[工学-材料科学与工程（可授工学、理学学位）] 0802[工学-机械工程] 0803[工学-光学工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：The authors received no specific funding for this study 

主　　题：Image fusion Res2Net-Transformer infrared image visible image 

摘      要：A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.