An Ensemble Co-Evolutionary based Algorithm for Classification Problems
Abstract
In this paper, the authors propose a dual-population co-evolutionary approach using ensemble learning approach (E-SOCA) to simultaneously solve both feature subset selection and optimal classifier design. Different from previous studies where each population retains only one best individual (Elite) after co-evolution, in this study, an elite community will be stored and calculated together through an ensemble learning algorithm to produce the final classification result. Experimental results on standard UCI problems with a variety of input features ranging from small to large sizes shows that the proposed algorithm results in more accuracy and stability than traditional algorithms.
References
G. Chandrashekar and F. Sahin, “A survey on feature selection methods,” Comput. Electr. Eng., vol. 40, no. 1, pp. 16–28, 2014.
Y. Saeys, I. Inza, and P. Larrañaga, “A review of feature selection techniques in bioinformatics,” Bioinformatics, vol. 23, no. 19, pp. 2507–2517, 2007.
M. Dash and H. Liu, “Feature selection for classification, Intelligent Data Analysis, vol. 1, no. 4, pp. 131–156, 1997.
H. Liu and Z. Zhao, “Manipulating data and dimension reduction methods: Feature selection,” in Encyclopedia of Complexity and Systems Science, pp. 5348–5359, Springer, 2009.
H. Liu, H. Motoda, R. Setiono, and Z. Zhao, “Feature selection: An ever evolving frontier in data mining,” in Feature Selection for Data Mining, vol. 10 of JMLR Proceedings, pp. 4–13, JMLR.org, 2010.
W. Siedlecki and J. Sklansky, “A note on genetic algorithms for largescale feature selection,” Pattern Recognition Letters, vol. 10, no. 5, pp. 335–347, 1989.
M. Shoorehdeli, M. Teshnehlab, and H. Moghaddam, “Feature subset selection for face detection using genetic algorithms and particle swarm optimization,” in IEEE International Conference on Networking, Sensing and Control (ICNSC), pp. 686–690, 2006.
D. Chen, K. Chan, and X. Wu, “Gene expression analyses using genetic algorithm based hybrid approaches,” in IEEE Congress on Evolutionary Computation, pp. 963–969, 2008.
F. Tan, X. Z. Fu, Y. Q. Zhang, and A. Bourgeois, “A genetic algorithmbased method for feature subset selection,” Soft Computing, vol. 12, no. 5, pp. 111–120, 2008.
L.-H. Chen and H.-D. Hsiao, “Feature selection to diagnose a business crisis by using a real ga-based support vector machine: An empirical study,” Expert Systems with Applications, vol. 35, no. 3, pp. 1145–1155, 2008.
J.-H. Hong and S.-B. Cho, “Efficient huge-scale feature selection with speciated genetic algorithm,” Pattern Recognition Letters, vol. 27, no. 2, pp. 143 – 150, 2006.
S. C. Yusta, “Different metaheuristic strategies to solve the feature selection problem,” Pattern Recognition Letters, vol. 30, pp. 525–534, 2009
S. F. da Silva, M. X. Ribeiro, J. do E.S. Batista Neto, C. Traina-Jr., and A. J. Traina, “Improving the ranking quality of medical image retrieval using a genetic feature selection method,” Decision Support Systems, vol. 51, no. 4, pp. 810–820, 2011.
R. Leardi, R. Boggia, and M. Terrile, “Genetic algorithms as a strategy for feature selection,” Journal of Chemometrics, vol. 6, no. 5, pp. 267– 281, 1992.
Y. Jeong, K. S. Shin, and M. K. Jeong, “An evolutionary algorithm with the partial sequential forward floating search mutation for largescale feature selection problems,” Journal of the Operational Research Society, pp. 1–10, 2014.
L. Santana, L. Silva, A. Canuto, F. Pintro, and K. Vale,“A comparative analysis of genetic algorithm and ant colony optimization to select attributes for an heterogeneous ensemble of classifiers,” in IEEE Congress on Evolutionary Computation, pp. 1–8, 2010.
J. Tian, M. Li, and F. Chen, “Dual-population based coevolutionary algorithm for designing RBFNN with feature selection,” Expert Syst. Appl., vol. 37, no. 10, pp. 6904–6918, 2010.
M. A. Potter and K. A. Jong, “A cooperative coevolutionary approach to function optimization,” pp. 249–257, 1994.
Lam Thu Bui, Van Truong VU, Dinh Thi Thu Huong,A novel evolutionary multi-objective ensemble learning approach for forecasting currency exchange rates. Data & Knowledge Engineering, 114, pp.40-66, 2018.
Van Truong VU, Lam Thu Bui, Trung Thanh Nguyen,A Coevolutionary approach for classification problems: Preliminary results, The 5th NAFOSTED Conference on Information and Computer Science (NICS), 2018.
T. T. H. Dinh, V. T. Vu, and T. L. Bui, “A multi-objective ensemble learning approach based on the non-dominated sorting differential evolution for forecasting currency exchange rates,” Proc. - 2016 8th Int. Conf. Knowl. Syst. Eng. KSE 2016, pp. 96–102, 2016.
J. H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. The MIT Press, Cambridge, USA, 1992.
ROKACH, Lior. Ensemble-based classifiers. Artificial Intelligence Review, 2010, 33.1-2: 1-39.
Y. Freund, R.E. Schapire, A desicion-theoretic generalizaton of on-line learning and an application to boosting, in:Proceedings of the European Conference on Computational Learning Theory, 1995, pp. 23–37.
Y. Freund, R.E. Schapire, Experiments with a new boosting algorithm, in: Icml, 1996, pp. 148–156.
R. Li, J. Lu, Y. Zhang, and T. Zhao, “Dynamic adaboost learning with feature selection based on parallel genetic algorithm for image annotation,” Knowledge-Based Systems, vol. 23, no. 3, pp. 195–201, 2010.