HMM-Based Photo-Realistic Talking Face Synthesis Using Facial Expression Parameter Mapping with Deep Neural Networks

作     者：Kazuki Sato Takashi Nose Akinori Ito 

作者机构：Department of Communication Engineering Graduate School of Engineering Tohoku University Sendai Japan 

出 版 物：《Journal of Computer and Communications》 (电脑和通信（英文）)

年 卷 期：2017年第5卷第10期

页      面：50-65页

学科分类：081203[工学-计算机应用技术] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：Visual-Speech Synthesis Talking Head Hidden Markov Models (HMMs) Deep Neural Networks (DNNs) Facial Expression Parameter 

摘      要：This paper proposes a technique for synthesizing a pixel-based photo-realistic talking face animation using two-step synthesis with HMMs and DNNs. We introduce facial expression parameters as an intermediate representation that has a good correspondence with both of the input contexts and the output pixel data of face images. The sequences of the facial expression parameters are modeled using context-dependent HMMs with static and dynamic features. The mapping from the expression parameters to the target pixel images are trained using DNNs. We examine the required amount of the training data for HMMs and DNNs and compare the performance of the proposed technique with the conventional PCA-based technique through objective and subjective evaluation experiments.