In previous section, we studied about Ensemble Models
In this short post we will just see the types of Ensemble models.
Types of Ensemble Models
- The above example is a very primitive type of ensemble model. There are better and statistically stronger ensemble methods that will yield better results
- Two most popular ensemble methodologies are
- Bagging
- Boosting
Bagging
- Take multiple boot strap samples from the population and build classifiers on each of the samples. For prediction take mean or mode of all the individual model predictions.
- Bagging has two major parts 1) Boot strap sampling 2) Aggregation of learners
- Bagging = Bootstrap Aggregating
- In Bagging we combine many unstable models to produce a stable model. Hence, the predictors will be very reliable(less variance in the final model).
Boot strapping
- We have a training data is of size N
- Draw random sample with replacement of size N – This gives a new dataset, it might have repeated observations, some observations might not have even appeared once.
- We are selecting records one-at-a-time, returning each selected record back in the population, giving it a chance to be selected again.
- Create B such new datasets. These are called boot strap datasets.
The next post is about the Bagging Algorithm.
