Devising optimal integration test orders using cost-benefit analysis

作     者：Fanyi MENG Ying WANG Hai YU Zhiliang ZHU Fanyi MENG;Ying WANG;Hai YU;Zhiliang ZHU

作者机构：Software CollegeNortheastern UniversityShenyang 110169China State Key Lab for Novel Software TechnologyNanjing UniversityNanjing 210023China 

出 版 物：《Frontiers of Information Technology & Electronic Engineering》 (信息与电子工程前沿（英文版）)

年 卷 期：2022年第23卷第5期

页      面：692-714页

核心收录：

学科分类：08[工学] 0837[工学-安全科学与工程] 0714[理学-统计学（可授理学、经济学学位）] 0701[理学-数学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 081202[工学-计算机软件与理论] 

基　　金：Project supported by the National Natural Science Foundation of China(Nos.61902056,61977014,and 61603082) the Shenyang Young and Middle-Aged Talent Support Program,China(No.ZX20200272) the Fundamental Research Funds for the Central Universities,China(No.N2017011) the Open Fund of State Key Lab for Novel Software Technology,Nanjing University,China(No.KFKT2021B01)。 

主　　题：Integration test order Cost–benefit analysis Probabilistic risk analysis Complex network 

摘      要：Integration testing is an integral part of software testing.Prior studies have focused on reducing test cost in integration test order generation.However,there are no studies concerning the testing priorities of critical classes when generating integration test orders.Such priorities greatly affect testing efficiency.In this study,we propose an effective strategy that considers both test cost and efficiency when generating test orders.According to a series of dynamic execution scenarios,the software is mapped into a multi-layer dynamic execution network(MDEN)model.By analyzing the dynamic structural complexity,an evaluation scheme is proposed to quantify the class testing priority with the defined class risk index.Cost–benefit analysis is used to perform cycle-breaking operations,satisfying two principles:assigning higher priorities to higher-risk classes and minimizing the total complexity of test stubs.We also present a strategy to evaluate the effectiveness of integration test order algorithms by calculating the reduction of software risk during their testing process.Experiment results show that our approach performs better across software of different scales,in comparison with the existing algorithms that aim only to minimize test cost.Finally,we implement a tool,ITOsolution,to help practitioners automatically generate test orders.