Research on Improving Rating Reliability for School-based Oral English Achievement Tests:The Design and Development of a Computer-aided Rating System

作     者：陆军 范劲松 

作者机构：上海交通大学外国语学院 

出 版 物：《Chinese Journal of Applied Linguistics》 (中国应用语言学（英文）)

年 卷 期：2011年第34卷第4期

页      面：59-71,128页

学科分类：0502[文学-外国语言文学] 050201[文学-英语语言文学] 0401[教育学-教育学] 05[文学] 

主　　题：school-based oral English achievement test rating reliability 

摘      要：Low rating reliability has long been the primary concern in school-based oral English achievement tests. In this study, a computer-aided rating system （CARS） was developed to improve inter- and intra-rater reliability through the instantiation of rating criteria, task division and random distribution, on-line training, reliability verification and sound wave ＂reading＂ and ＂writing.＂ A rating experiment was conducted among six raters to compare intra- and inter-rater reliability between traditional rating and rating with CARS. At the end of each round of rating, a conference was held. Both quantitative an~ qualitative analyses show that CARS can significantly improve inter- and intra-rater reliability, mainly through helping raters use criteria more accurately and focus more attention on rating. In addition, the research has also shed light upon further study on improving rating reliability.