1 / 16

Popular Ensemble Methods: An Empirical Study

Popular Ensemble Methods: An Empirical Study. David Opitz and Richard Maclin Presented by Scott Wespi 5/22/07. Outline. Ensemble methods Classifier Ensembles Bagging vs Boosting Results Conclusion. Ensemble Methods.

giolla
Download Presentation

Popular Ensemble Methods: An Empirical Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Popular Ensemble Methods: An Empirical Study David Opitz and Richard Maclin Presented by Scott Wespi 5/22/07

  2. Outline • Ensemble methods • Classifier Ensembles • Bagging vs Boosting • Results • Conclusion

  3. Ensemble Methods • Sets of individually trained classifiers whose predictions are combined when classifying new data • Bagging (1996) • Boosting (1996) • How are bagging and boosting influenced by the learning algorithm? • Decision trees • Neural networks *Note: Paper is from 1999

  4. Classifier Ensembles • Goal: highly accurate individual classifiers that disagree as much as possible • Bagging and boosting create disagreement

  5. Bagging vs. Boosting

  6. Ada-Boosting vs Arcing • Ada-Boosting • Every sample has 1/N weight initially, increases every time sample was skipped or misclassified • Arcing • If mi = number of times ith example was misclassified

  7. Neural Networks • Ada-Boosting • Arcing • Bagging • White bar represents 1 • standard deviation

  8. Decision Trees

  9. Composite Error Rates

  10. Neural Networks: Bagging vs Simple

  11. Ada-Boost: Neural Networks vs. Decision Trees • NN • DT • Box represents • reduction in error

  12. Arcing

  13. Bagging

  14. Noise • Hurts boosting the most

  15. Conclusions • Performance depends on data and classifier • In some cases, ensembles can overcome bias of component learning algorithm • Bagging is more consistent than boosting • Boosting can give much better results on some data

More Related