Two simple enhancements to tree-based methods are called bagging and boosting. These iterative schemes each produce a committee of expert tree models by resampling with replacement from the initial data set. Afterward, the expert tree models are averaged using a plurality voting scheme if the response is discrete, or simple averaging if the response is continuous. The difference between bagging and boosting is the way in which data are resampled. In the former, all observations have equal probability of entering the next bootstrap sample; in the latter, problematic observations (i.e., observations that have been frequently misclassified) have a higher probability of selection.
Was this article helpful?