Multi-scale joint feature network for micro-expression recognition

作     者：Xinyu Li Guangshun Wei Jie Wang Yuanfeng Zhou Xinyu Li;Guangshun Wei;Jie Wang;Yuanfeng Zhou

作者机构：School of SoftwareShandong UniversityJinan 250101China 

出 版 物：《Computational Visual Media》 (计算可视媒体（英文版）)

年 卷 期：2021年第7卷第3期

页      面：407-417页

核心收录：

学科分类：08[工学] 080203[工学-机械设计及理论] 0802[工学-机械工程] 

基　　金：supported by the NSFC–Zhejiang Joint Fund of the Integration of Informatization and Industrialization under Grant No.U1909210 the the National Natural Science Foundation of China under Grant No.61772312 the Fundamental Research Funds of Shandong University(Grant No.2018JC030) 

主　　题：micro-expression recognition multi-scale feature optical flow deep learning 

摘      要：Micro-expression recognition is a substantive cross-study of psychology and computer science,and it has a wide range of applications(e.g.,psychological and clinical diagnosis,emotional analysis,criminal investigation,etc.).However,the subtle and diverse changes in facial muscles make it difficult for existing methods to extract effective features,which limits the improvement of micro-expression recognition ***,we propose a multi-scale joint feature network based on optical flow images for micro-expression ***,we generate an optical flow image that reflects subtle facial motion *** optical flow image is then fed into the multi-scale joint network for feature extraction and *** proposed joint feature module(JFM)integrates features from different layers,which is beneficial for the capture of micro-expression features with different *** improve the recognition ability of the model,we also adopt a strategy for fusing the feature prediction results of the three JFMs with the backbone *** experimental results show that our method is superior to state-of-the-art methods on three benchmark datasets(SMIC,CASME II,and SAMM)and a combined dataset(3 DB).