1 / 17

Dirichlet process tutorial

Dirichlet process tutorial. Bryan Russell. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A. Goals. Intuitive understanding of Dirichlet processes and applications Minimize math and maximize pictures

amadis
Download Presentation

Dirichlet process tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Dirichlet process tutorial Bryan Russell TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA

  2. Goals • Intuitive understanding of Dirichlet processes and applications • Minimize math and maximize pictures • Motivate you to go through the math to understand implementation

  3. Disclaimers • What I’m about to tell you applies more generally • We’ll gloss over lots of math (especially measure theory); look at the original papers for details

  4. What’s this good for? • Principled, Bayesian method for fitting a mixture model with an unknown number of clusters • Because it’s Bayesian, can build hierarchies (e.g. HDPs) and integrate with other random variables in a principled way

  5. Aren’t there other ways to count the number of clusters?

  6. Gaussian mixture model, revisited

  7. Gaussian mixture model, revisited

  8. Gaussian mixture model, revisited

  9. Let us generate data points…

  10. Multinomial weights: prior probabilities of the mixtures

  11. For each data point, choose cluster center h

  12. Generate points x from the Gaussian mixture h

  13. Let us be more Bayesian… Put a prior over mixture parameters For Gaussian mixtures, this is a normal inverse-Wishart density

  14. Suppose we do not know the number of clusters We could sample Gaussian parameters for each data point However, the parameters may all be unique, i.e. there is one Gaussian mixture for each data point--overfitting

  15. Dirichlet processes to the rescue Draws from Dirichlet processes have a nice clustering property: Normal inverse-Wishart density Concentration parameter is a density over the parameters and is discrete with probability one

  16. Visualizing Dirichlet process draws Think of these as prior weights over the parameters

  17. DP mixture model This model has a bias to “bunch” parameters together. The concentration parameter controls this “bunching” property: lower values will find fewer clusters and vice versa for higher values

More Related