Face anti-spoofing based on multi-modal and multi-scale features fusion

作     者：Kong Chao Ou Weihua Gong Xiaofeng Li Weian Han Jie Yao Yi Xiong Jiahao Kong Chao;Ou Weihua;Gong Xiaofeng;Li Weian;Han Jie;Yao Yi;Xiong Jiahao

作者机构：School of Big Data and Computer ScienceGuizhou Normal UniversityGuiyang 550025China Special Key Laboratory of Artificial Intelligence and Intelligent Control of Guizhou ProvinceGuizhou Institute of TechnologyGuiyang 550003China Guizhou Science and Technology Information CenterDepartment of Science and Technology of Guizhou ProvinceGuiyang 550002China 

出 版 物：《The Journal of China Universities of Posts and Telecommunications》 (中国邮电高校学报（英文版）)

年 卷 期：2022年第29卷第6期

页      面：73-82页

核心收录：

学科分类：08[工学] 080203[工学-机械设计及理论] 0802[工学-机械工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Natural Science Foundation of China(61962010,62262005) the Natural Science Foundation of Guizhou Priovince(QianKeHeJichu1425). 

主　　题：face anti-spoofing multi-modal fusion multi-scale fusion self-attention network(SAN) feature pyramid network(FPN) 

摘      要：Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network(CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion(MMFF) is proposed. Specifically, first residual network(Resnet)-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network(FPN), finally squeeze-and-excitation fusion(SEF) module and self-attention network(SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.