Using Confidence Interval to Summarize the EvaluatingResults of DSM Systems

作     者：施巍松 唐志敏 施劲松 

作者机构：InstituteofComputingTechnologyChineseAcademyofSciencesBeijing100080P.R.China 

出 版 物：《Journal of Computer Science & Technology》 (计算机科学技术学报（英文版）)

年 卷 期：2000年第15卷第1期

页      面：73-83页

核心收录：

学科分类：08[工学] 081202[工学-计算机软件与理论] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：国家自然科学基金 国家攀登计划项目 

主　　题：data-summarizing technique performance evaluation DSM system confidence interval benchmarking 

摘      要：Distributed Shared Memory (DSM) systems have gained popularacceptance by combining the scalability and low cost of distributed system with theease of use of single address space. Many new hardware DSM and software DSMsystems have been proposed in recent years. In general, benchmarking is widely usedto demonstrate the performance advantages of new systems. However, the commonmethod used to summarize the measured results is the arithmetic mean of ratios,which is incorrect in some cases. Furthermore, many published papers list a lot ofdata only, and do not summarize them effectively, which confuse users greatly. Infact, many users want to get a single number as conclusion, which is not providedin old summarizing techniques. Therefore, a new data-summarizing technique basedon confidence interval is proposed in this paper. The new technique includes twodata-summarizing methods: (1) paired confidence interval method; (2) unpairedconfidence interval method. With this new technique, it is concluded that at someconfidence one system is better than others. Four examples are shown to demonstratethe advantages of this new technique. Furthermore, with the help of confidence level,it is proposed to standardize the benchmarks used for evaluating DSM systems sothat a convincing result can be got. In addition, the new summarizing technique fitsnot only for evaluating DSM systems, but also for evaluating other systems, such asmemory system and communication systems.