1 / 58

THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 600N:  Reasoning and Decision under Uncertainty Summer 2010

THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 600N:  Reasoning and Decision under Uncertainty Summer 2010 Nevin L. Zhang Room 3504, phone: 2358-7015, Email: lzhang@cs.ust.hk Home page PMs for Classification PMs for Clustering: Continuous data PMs for Clustering: Discrete data

issac
Download Presentation

THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 600N:  Reasoning and Decision under Uncertainty Summer 2010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGYCSIT 600N:  Reasoning and Decision under Uncertainty Summer 2010 Nevin L. ZhangRoom 3504, phone: 2358-7015, Email: lzhang@cs.ust.hkHome page

  2. PMs for Classification PMs for Clustering: Continuous data PMs for Clustering: Discrete data L09: Probabilistic Models (PMs) for Classification and Clustering

  3. The problem: Given data: Find mapping (A1, A2, …, An) |- C Possible solutions ANN Decision tree (Quinlan) … (SVM: Continuous data) Classification

  4. Probabilistic Approach to Classification

  5. Will Boss Play Tennis?

  6. Will Boss Play Tennis?

  7. Bayesian Networks for Classification • Naïve Bayes model often has good performance in practice • Drawbacks of Naïve Bayes: • Attributes mutually independent given class variable • Often violated, leading to double counting. • Fixes: • General BN classifiers • Tree augmented Naïve Bayes (TAN) models • …

  8. Bayesian Networks for Classification • General BN classifier • Treat class variable just as another variable • Learn a BN. • Classify the next instance based on values of variables in the Markov blanket of the class variable. • Pretty bad because it does not utilize all available information because of Markov boundary

  9. Bayesian Networks for Classification • Tree-Augmented Naïve Bayes (TAN) model • Capture dependence among attributes using a tree structure. • During learning, • First learn a tree among attributes: use Chow-Liu algorithm • Special structure learning problem, easy • Add class variable and estimate parameters • Classification • arg max_c P(C=c|A1=a1, …, An=an) • BN inference

  10. PMs for Classification PMs for Clustering: Continuous data • Gaussian distributions • Parameter estimation for Gaussian distributions • Gaussian mixtures • Learning Gaussian mixtures PMs for Clustering: Discrete data Outline

  11. http://www-stat.stanford.edu/~naras/jsm/NormalDensity/NormalDensity.htmlhttp://www-stat.stanford.edu/~naras/jsm/NormalDensity/NormalDensity.html • Real-world example of Normal Distributions?

  12. Bivariate Gaussian Distribution

  13. Bivariate Gaussian Distribution

  14. PMs for Classification PMs for Clustering: Continuous data • Gaussian distributions • Parameter estimation for Gaussian distributions • Gaussian mixtures • Learning Gaussian mixtures PMs for Clustering: Discrete data Outline

  15. Data: Example Mean vector Covariance Matrix

  16. PMs for Classification PMs for Clustering: Continuous data • Gaussian distributions • Parameter estimation for Gaussian distributions • Gaussian mixtures • Learning Gaussian mixtures PMs for Clustering: Discrete data Outline

  17. PMs for Classification PMs for Clustering: Continuous data • Gaussian distributions • Parameter estimation for Gaussian distributions • Gaussian mixtures • Learning Gaussian mixtures PMs for Clustering: Discrete data Outline

  18. Learning Gaussian Mixture Models

  19. MLE

  20. http://www.socr.ucla.edu/Applets.dir/MixtureEM.html

  21. PMs for Classification PMs for Clustering: Continuous data PMs for Clustering: Discrete data L09: Probabilistic Models (PMs) for Classification and Clustering

  22. PMs for Classification PMs for Clustering: Continuous data PMs for Clustering: Discrete data • A generalization L09: Probabilistic Models (PMs) for Classification and Clustering

More Related