Task-Oriented Semantic Communication with Foundation Models

作     者：Chen Mingkai Liu Minghao Zhang Zhe Xu Zhiping Wang Lei Chen Mingkai;Liu Minghao;Zhang Zhe;Xu Zhiping;Wang Lei

作者机构：Key Laboratory of Broadband Wireless Communication and Sensor Network TechnologyMinistry of EducationNanjing University of Posts and TelecommunicationsNanjing 210003China School of Ocean Information EngineeringJimei UniversityXiamen 361021China Key Laboratory of Underwater Acoustic Communication and Marine Information Technology(Xiamen University)Ministry EducationXiamen 361005China 

出 版 物：《China Communications》 (中国通信（英文版）)

年 卷 期：2024年第21卷第7期

页      面：65-77页

核心收录：

学科分类：080904[工学-电磁场与微波技术] 0810[工学-信息与通信工程] 0809[工学-电子科学与技术（可授工学、理学学位）] 08[工学] 080402[工学-测试计量技术及仪器] 0804[工学-仪器科学与技术] 081001[工学-通信与信息系统] 

基　　金：supported in part by the National Natural Science Foundation of China under Grant(62001246,62231017,62201277,62071255) the Natural Science Foundation of Jiangsu Province under Grant BK20220390 Key R and D Program of Jiangsu Province Key project and topics under Grant(BE2021095,BE2023035) the Natural Science Research Startup Foundation of Recruiting Talents of Nanjing University of Posts and Telecommunications(Grant No.NY221011) National Science Foundation of Xiamen,China(No.3502Z202372013) Open Project of the Key Laboratory of Underwater Acoustic Communication and Marine Information Technology(Xiamen University)of the Ministry of Education,China(No.UAC202304) 

主　　题：diffusion model foundation model joint source-channel coding task-oriented semantic communication 

摘      要：In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the *** rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication *** this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation ***,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation ***,we have differentiated semantic compression and transmission approaches for segmented ***,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users specific task ***,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.