1 / 9

COMP 4332 Tutorial 4 Mar 1 Yin Zhu yinz@cset.hk

Ensemble heterogeneous models . COMP 4332 Tutorial 4 Mar 1 Yin Zhu yinz@cse.ust.hk. Ensemble techniques for heterogeneous classifiers. Greedy search/ensemble selection Stacking More: Sam Reid, A Review of  Heterogeneous Ensemble Methods , 2007. . Ensemble iff accurate & diverse .

nia
Download Presentation

COMP 4332 Tutorial 4 Mar 1 Yin Zhu yinz@cset.hk

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ensemble heterogeneous models COMP 4332 Tutorial 4 Mar 1 Yin Zhu yinz@cse.ust.hk

  2. Ensemble techniques for heterogeneous classifiers • Greedy search/ensemble selection • Stacking • More: • Sam Reid, A Review of Heterogeneous EnsembleMethods, 2007.

  3. Ensemble iff accurate & diverse

  4. Ensemble selection • Caruanaet. al

  5. Three variants to avoid over-fitting • Selection with replacement • Sorted ensemble initialization • Bagged ensemble selection

  6. Good f()? • Many (good) classifiers are fine for f(). • Particularly, Gradient Boosted Trees: • Software: • R’s gbm (tree-based functional gradient descent boosting) • RT-Rank in C++. • TreeNet by Salford Systems. (commercial software) • Used by: • KDDCUP 2009, Winners slow track: University of Melbourne. • 2nd place of Netflix challenge, the Ensemble team, link • Reference: • Friedman, Greedy function approximation: a gradient boosting machine, 1999.

  7. Reference on ensemble in general • Tom Dietterich. • Ensemble Methods in Machine Learning, 2000. PS file. • An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization, 2000. Springer Link. • Leo Breiman, Bagging predictors. 1996. Springer Link. • Freund & Schapire, Experiments with a New Boosting Algorithm. 1996. PDF. • Z.-H. Zhou. Ensemble Methods: Foundations and Algorithms, Boca Raton, FL: Chapman & Hall/CRC, 2012. (ISBN 978-1-439-830031)  [TOC; Sample chapters: Chapter 2, Chaper 6]

More Related