检索结果-南通市图书馆

LMR-CBT: learning modality-fused representations with CB-Transformer for multimodal emotion recognition from unaligned multimodal sequences

引用

Frontiers of Computer Science 2024年第4期18卷 39-47页

作者： Ziwang FU Feng LIU Qing XU Xiangling FU Jiayin QI School of Computer Science(National Pilot Software Engineering School) Beijing University of Posts and TelecommunicationsBeijing 100876China Key Laboratory of Trustworthy Distributed Computing and Service(BUPT) Beijing 100876China Shanghai International School of Chief Technology Officer East China Normal UniversityShanghai 200062China School of Computer Science and Technology East China Normal UniversityShanghai 200062China School of Cyberspace Security Guangzhou UniversityGuangdong 510006China

Learning modality-fused representations and processing unaligned multimodal sequences are meaningful and challenging in multimodal emotion *** approaches use directional pairwise attention or a message hub to fuse language,visual,and audio ***,these fusion methods are often quadratic in complexity with respect to the modal sequence length,bring redundant information and are not *** this paper,we propose an efficient neural network to learn modality-fused representations with CB-Transformer(LMR-CBT)for multimodal emotion recognition from unaligned multi-modal ***,we first perform feature extraction for the three modalities respectively to obtain the local structure of the ***,we design an innovative asymmetric transformer with cross-modal blocks(CB-Transformer)that enables complementary learning of different modalities,mainly divided into local temporal learning,cross-modal feature fusion and global self-attention *** addition,we splice the fused features with the original features to classify the emotions of the ***,we conduct word-aligned and unaligned experiments on three challenging datasets,IEMOCAP,CMU-MOSI,and *** experimental results show the superiority and efficiency of our proposed method in both *** with the mainstream methods,our approach reaches the state-of-the-art with a minimum number of parameters.

关键词： modality-fused representations cross-model blocks multimodal emotion recognition unaligned multimodal sequences computational affection

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

multimodal emotion recognition in the metaverse era:New needs and transformation in mental health work

引用

World Journal of Clinical Cases 2024年第34期12卷 6674-6678页

作者： Yan Zeng Jun-Wen Zhang Jian Yang Department of Psychology The Second Affiliated Hospital of Chongqing Medical UniversityChongqing 400010China Department of Gastroenterology The First Affiliated Hospital of Chongqing Medical UniversityChongqing 400016China

This editorial comments on an article recently published by López del Hoyo et *** metaverse,hailed as"the successor to the mobile Internet",is undoubtedly one of the most fashionable terms in recent *** metaverse development is a complex and multifaceted evolutionary process influenced by many factors,it is almost certain that it will significantly impact our lives,including mental health *** any other technological advancements,the metaverse era presents a double-edged sword for mental health work,which must clearly understand the needs and transformations of its target *** this editorial,our primary focus is to contemplate potential new needs and transformation in mental health work during the metaverse era from the pers-pective of multimodal emotion recognition.

关键词： multimodal emotion recognition Metaverse Needs Transformation Mental health

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

TACFN:Transformer-Based Adaptive Cross-Modal Fusion Network for multimodal emotion recognition

引用

CAAI Artificial Intelligence Research 2023年第1期2卷 75-82页

作者： Feng Liu Ziwang Fu Yunlong Wang Qijian Zheng School of Computer Science and Technology East China Normal UniversityShanghai 200062China MTlab Meitu(China)LimitedBeijing 100876China Institute of Acoustics University of Chinese Academy of SciencesBeijing 100084China

The fusion technique is the key to the multimodal emotion recognition ***,cross-modal attention-based fusion methods have demonstrated high performance and strong ***,cross-modal attention suffers from redundant features and does not capture complementary features *** find that it is not necessary to use the entire information of one modality to reinforce the other during cross-modal interaction,and the features that can reinforce a modality may contain only a part of *** this end,we design an innovative Transformer-based Adaptive Cross-modal Fusion Network(TACFN).Specifically,for the redundant features,we make one modality perform intra-modal feature selection through a self-attention mechanism,so that the selected features can adaptively and efficiently interact with another *** better capture the complementary information between the modalities,we obtain the fused weight vector by splicing and use the weight vector to achieve feature reinforcement of the *** apply TCAFN to the RAVDESS and IEMOCAP *** fair comparison,we use the same unimodal representations to validate the effectiveness of the proposed fusion *** experimental results show that TACFN brings a significant performance improvement compared to other methods and reaches the state-of-the-art *** code and models could be accessed from https://***/shuzihuaiyu/TACFN.

关键词： multimodal emotion recognition multimodal fusion adaptive cross-modal blocks Transformer computational perception

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：