Improving Parallel Corpus Quality for Chinese-Vietnamese Statistical Machine Translation

作     者：Huu-anh Tran Yuhang Guo Ping Jian Shumin Shi Heyan Huang 

作者机构：Department of Computer Science and TechnologyBeijing Institute of Technology Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing ApplicationBeijing Institute of Technology 

出 版 物：《Journal of Beijing Institute of Technology》 (北京理工大学学报（英文版）)

年 卷 期：2018年第27卷第1期

页      面：127-136页

核心收录：

学科分类：081203[工学-计算机应用技术] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Supported by the National Basic Research Program of China(973Program)(2013CB329303) the National Natural Science Foundation of China(61502035) 

主　　题：parallel corpus filtering low resource languages bilingual movie subtitles machine translation Chinese Vietnamese translation 

摘      要：The performance of a machine translation system heavily depends on the quantity and quality of the bilingual language resource. However,getting a parallel corpus,which has a large scale and is of high quality,is a very difficult task especially for low resource languages such as Chinese-Vietnamese. Fortunately,multilingual user generated contents（ UGC）,such as bilingual movie subtitles,provide us access to automatic construction of the parallel corpus. Although the amount of UGC parallel corpora can be considerable,the original corpus is not suitable for statistical machine translation（ SMT） systems. The corpus may contain translation errors,sentence mismatching,free translations,etc. To improve the quality of the bilingual corpus for SMT systems,three filtering methods are proposed： sentence length difference,the semantic of sentence pairs,and machine learning. Experiments are conducted on the Chinese to Vietnamese translation *** results demonstrate that all the three methods effectively improve the corpus quality,and the machine translation performance（ BLEU score） can be improved by 1. 32.