A novel approach for regularization of ensemble learning in classification and regression analysis

Ensemble method requires in classification or regression model, to obtain better accuracy performance. It uses multiple learning algorithms, for which it can easily approaches to a better classifier or predictor compared to a single model. Gradient Boosting is a machine learning technique that can be used for both classification and regression problems. Basically this method is applied to enhance the performance of decision tree models. It builds a model in a step wise manner by optimizing a differential loss function or cost function. The main objective of this paper is to adjust the parameters of Gradient Boosting to avoid the problem of over fitting. This technique is generally called as regularization and the problem of over fitting can be solved by two regularization methods such as shrinkage and sub-sampling. In this work, those two regularization methods have been explored with the Gradient Boosting for both the classification and regression problems. The results have been found to be very interesting while compared with non regularized methods.