1 / 24

Attribute Learning for Understanding Unstructured Social Activity

Attribute Learning for Understanding Unstructured Social Activity. Yanwei Fu, Timothy M. Hospedales , Tao Xiang, and Shaogang Gong School of EECS, Queen Mary University of London, UK Presented by Amr El-Labban VGG Reading Group, Dec 5 th 2012 . Contributions.

avani
Download Presentation

Attribute Learning for Understanding Unstructured Social Activity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Attribute Learning for Understanding Unstructured Social Activity Yanwei Fu, Timothy M. Hospedales, Tao Xiang, and ShaogangGong School of EECS, Queen Mary University of London, UK Presented by Amr El-Labban VGG Reading Group, Dec 5th 2012

  2. Contributions • Unstructured social activity attribute (USAA) dataset • Semi-latent attribute space • Topic model based attribute learning

  3. Objective • Automatic classification of unstructured group social activity • Use an attribute based approach • Start with sparse, user defined attributes • Add latent ones • Learn jointly

  4. Dataset • 1500 videos, 8 classes • 69 visual/audio attributes manually labelled • Weak labelling • SIFT, STIP and MFCC features used • Data available (features, attributes, YouTube IDs)

  5. Semi-Latent Attribute Space • Space consisting of: • User defined attributes • Discriminative latent attributes • Non-discriminative (background) latent attributes

  6. Topic modelling d y d x x y = P(y|d) P(x|d) P(x|y) x – low level features (‘words’) y – attributes (‘topics’) d – ‘documents’

  7. Latent Dirichlet Allocation y x x – low level features y – attributes (user defined and latent) θ – attribute distribution φ – word distribution α, β – Dirichlet parameters

  8. Aside: Dirichletdisribution • Distribution over multinomial distributions • Parameterised by α α = (6,2,2) α = (3,7,5) α = (2,3,4) α = (6,2,6)

  9. Aside: Dirichletdisribution • Important things to know: • - peak is closer to larger values • - large gives small variance • <1 gives more sparse distributions

  10. Latent Dirichlet Allocation y x x – low level features y – attributes (user defined and latent) θ – attribute distribution φ – word distribution α, β – Dirichlet parameters

  11. Latent Dirichlet Allocation y x Generative model for each document: Choose θ ~ Dir(α) Choose φ ~ Dir(β) for each word: Choose y ~ Multinomial(θ) Choose x ~ Multinomial(φy)

  12. Latent Dirichlet Allocation y x

  13. Latent Dirichlet Allocation y x • EM to learn Dirichlet parameters: • Approximate inference for posterior:

  14. SLAS • User defined part • Per instance prior on α. • Set to zero when attribute isn’t present in ground truth • Latent part • First half “class conditional” • One α per class. • All but one constrained to zero. • Second half “background” • Unconstrained

  15. Classification • Use SLAS posterior to map from raw data to attributes • Use standard classifier (logistic regression) from attributes to classes

  16. N-shot transfer learning • Split data into two partitions – source and target • Learn attribute models on source data • Use N examples from target to learn attribute-class mapping

  17. Zero-shot learning • Detect novel class • Manually defined attribute-class “prototype” • Improve with self-training algorithm: • Infer attributes for novel data • NN matching in user defined space against protoype • For each novel class: • Find top K matches • Train new prototype in full attribute space (mean of top K) • NN matching in the full space

  18. Experiments • Compare three models: • Direct: KNN or SVM on raw data • SVM-UD+LR: SVM to map raw data to attributes, LR maps attributes to classes • SLAS+LR: SLAS to map raw data to attributes, LR learns classes based on user-defined and class conditional attributes.

  19. MASSIVE HACK • “The UD part of the SLAS topic profile is estimating the same thing as the SVM attribute classifiers, however the latter are slightly more reliable due to being discriminatively optimised. As input to LR, we therefore actually use the SVM attribute classier outputs in conjunction with the latent part of our topic profile.”

  20. Results - classification • SLAS+LR better as number if training data and user defined attributes decreases • Copes with 25% wrong attribute bits

  21. Results - classification • KNN and SVM have vertical bands – consistent misclassification

  22. Results – N-shot transfer learning • Vary number of user defined attributes • SVM+LR cannot cope with zero attributes

  23. Results – Zero-shot transfer learning • Two cases: • Continuous prototype – mean attribute profile • Binary prototype – thresholded mean • Tested without background latent attributes (SLAS(NF))

  24. Conclusion • Augmenting SVM and user defined attributes with latent ones definitely helps. • Experimental hacks make it hard to say how good the model really is…

More Related