Improving bagging performance through multi-algorithm ensembles

作     者：Kuo-Wei HSU Jaideep SRIVASTAVA 

作者机构：Department of Computer Science Chengchi University Taipei 11605 China Department of Computer Science and Engineering University of Minnesota Minneapolis MN 55455 USA 

出 版 物：《Frontiers of Computer Science》 (中国计算机科学前沿（英文版）)

年 卷 期：2012年第6卷第5期

页      面：498-512页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 07[理学] 0701[理学-数学] 070101[理学-基础数学] 

主　　题：bagging classification diversity ensemble 

摘      要：Working as an ensemble method that establishes a committee of classifiers first and then aggregates their out- comes through majority voting, bagging has attracted consid- erable research interest and been applied in various applica- tion domains. It has demonstrated several advantages, but in its present form, bagging has been found to be less accurate than some other ensemble methods. To unlock its power and expand its user base, we propose an approach that improves bagging through the use of multi-algorithm ensembles. In a multi-algorithm ensemble, multiple classification algorithms are employed. Starting from a study of the nature of diver- sity, we show that compared to using different training sets alone, using heterogeneous algorithms together with different training sets increases diversity in ensembles, and hence we provide a fundamental explanation for research utilizing het- erogeneous algorithms. In addition, we partially address the problem of the relationship between diversity and accuracy by providing a non-linear function that describes the relation- ship between diversity and correlation. Furthermore, after re- alizing that the bootstrap procedure is the exclusive source of diversity in bagging, we use heterogeneity as another source of diversity and propose an approach utilizing heterogeneous algorithms in bagging. For evaluation, we consider several benchmark data sets from various application domains. The results indicate that, in terms of Fl-measure, our approach outperforms most of the other state-of-the-art ensemble meth- ods considered in experiments and, in terms of mean margin, our approach is superior to all the others considered in experiments.