A novel ensemble learning method for classification and regression based on weighted stacking of estimators

In machine learning, ensemble methods are emerging and powerful strategies to improve the robustness and accuracy of both the supervised and unsupervised learning solutions. The basic principle of ensemble strategy is to work on a combination of diversified base classifiers and regerssors to strengthen the both kind of weak classifiers and regressors and are available for a single task, merging all of the results which leads to better performance of the classifiers and regressors. In this study, an attempt has been made to implement study and analyze the importance of ensemble strategies on either classification and regression techniques to predict the class label or future value respectively. Here, for experimentation two sets of different datasets are used for both of the tasks and the proposed estimator based on weighted staking has been experimented and evaluated in two levels using RF, a variant of ANN i.e. Multi-Layer Probabilistic Neural Network (MLPNN), a lazy learner k-NN if first level of experimentation to get the individual predictions of those heterogeneous models and the in second level of experimentation, the weighted staking ensemble leaning has been experimented with SVM and the performance of the all the individual models along with stacked SVM have been evaluated using various performance matrices such as; specificity, sensitivity and F-score as well as T-test based statistical analysis has been considered to accept the performance of the proposed model.