1 / 24

Unsupervised Transfer Classification

Unsupervised Transfer Classification. Application to Text Categorization Tianbao Yang , Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University. Overview. Introduction Related Work Unsupervised Transfer Classification Problem Definition Approach & Analysis Experiments

rosajohnson
Download Presentation

Unsupervised Transfer Classification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University

  2. Overview • Introduction • Related Work • Unsupervised Transfer Classification • Problem Definition • Approach & Analysis • Experiments • Conclusions

  3. Introduction supervised semi-supervised • Classification: • supervised learning • semi-supervised learning • What if No label information is available? • impossible but not with some additional information unsupervised classification

  4. Introduction • Unsupervised transfer classification (UTC) • a collection of training examples and their assignments to auxiliary classes • to build a classification model for a target class auxiliary class 1 auxiliary class K …. conditional probabilities No Labeled training examples prior target class

  5. Introduction: Motivated Examples Image Annotation Social Tagging target classes auxiliary classes google phone verizon apple sky sun water grass 1 0 ? 0 ? 0 1 0 1 1 ? 0 1 1 0 ? 1 0 1 ? 0 ? 1 1 0 0 ? 1 ? 0 0 1 / / / ? / / / ? How to predict an annotation word/social tag that does not appear in the training data ?

  6. Related Work • Transfer Learning • transfer knowledge from source domain to target domain • similarity: transfer label information for auxiliary classes to target class • difference: assume NO label information for target class • Multi-Label Learning, Maximum Entropy Model

  7. Unsupervised Transfer Classification Data for auxiliary class Examples assignments to auxiliary classes Auxiliary Classes Goal target class target class label target classification model Class Information Prior probability conditional probabilities

  8. Maximum Entropy Model (MaxEnt) Favor uniform distribution Feature statistics computed from conditional model Feature statistics computed from training data : the jth feature function

  9. Generalized MaxEnt Equality constraints With a large probability Inequality constraints

  10. Generalized MaxEnt

  11. Generalized MaxEnt is unknown for target class How to extend generalized MaxEnt to unsupervised transfer classification ?

  12. Unsupervised Transfer Classification • Estimating feature statistics of target class from those of the auxiliary classes ~ ~

  13. Unsupervised Transfer Classification • Build up Relation between Auxiliary Classes and Target Class Independence Assumption

  14. Unsupervised Transfer Classification • Estimating feature statistics for the target class by regression Feature Statistics for Auxiliary Classes Class Information Feature Statistics for Target Class

  15. Unsupervised Transfer Classification • Dual problem : function of U; definition can be found in paper

  16. Consistency Result With a large probability The dual solution obtained by the proposed approach The optimal dual solution using the label information for the target class

  17. Experiments • Text categorization • Data sets: multi-labeled data • Protocol: leave one-class out as the target class • Metric: AUC (Area under ROC curve)

  18. Experiments: Baselines • cModel • train a classifier for each auxiliary class • linearly combine them for the target class • cLabel • predict the assignment of the target class for training examples by linearly combining the labels of auxiliary classes • train a classifier using the predicted labels for target class • GME-avg • use generalized maxent model • compute the feature statistics for the target class by linearly combining those for the auxiliary classes • Proposed Approach: GME-Reg

  19. Experiment (I) • Estimate class information from training data

  20. Experiment (I) • Estimate class information from training data • Compare to the classifier of the target class learned by supervised learning 1500 2500

  21. Experiment (II) • Obtain class information from external sources • Datasets: bibtex and delicious • bibsonomy www.bibsonomy.org/tagsbibtex • ACM DL www.portal.acm.orgbibtex • deli.cio.us www.delicious.com/tagdelicious

  22. Experiment (II) • Comparison with Supervised Classification 650 1000~1200

  23. Conclusions • A new problem: unsupervised transfer classification • A statistical framework for unsupervised transfer classification • based on generalized maximum entropy • robust estimate feature statistics for target class • provable performance by consistency analysis • Future Work • relax independence assumption • better estimation of feature statistics for target class

  24. Thanks Questions ?

More Related