The Construction of a Majority-Voting Ensemble Based on the Interrelation and Amount of Information of Features

Abstract

In this paper, we introduced a new ensemble learning algorithm called VIBES, which is better in terms of performance when compared to 85 machine learning algorithms in WEKA tool. This new algorithm is based on three major processes: (i) making an assumption regarding whether features are dependent on or independent of each other, (ii) computing the amount of information of features when it is assumed that they are dependent on each other and then sorting them in a descending manner based on the amount of information, (iii) speeding up the algorithm by optimizing the forward search algorithm that is used in the construction of the final hypothesis from base learner hypotheses. As a result of these processes, it has been seen in the experiments that choosing the relevant assumption can boost learning performance if features are independent of each other; considering features according to the amount of information provides high accuracy and diversity of base learner models. According to experiment results, the algorithm that has been developed has the highest average classification accuracy rate across the 33 datasets. The highest and the lowest average classification accuracy rates have been found to be 89.80 and 78.03 %, respectively.