Discriminative Segment Annotation in Weakly Labeled Video

1. Discriminative Segment Annotation in Weakly Labeled Video Kevin Tang, Rahul Sukthankar Appeared in CVPR 2013 (Oral)

2. Research Problem • Input: a weakly labeled video (eg., “dog”) • Output: identify segments that correspond to the label to generate the semantic segmentation, i.e., classify each segment either as coming from concept “dog” (called concept segments), or not (called background segments). • Pipeline • Perform unsupervised spatiotemporal segmentation. • Propose an algorithm to identify the meaningful segment.

3. Contributions • Present a interpretation framework to analyze a broad class of existing weakly supervised learning algorithms about segment annotation problem. • Propose a discriminative algorithm CRANE (Concept Ranking According to Negative Exemplars) for segment annotation.

4. Interpretation framework • Pairwise distance matrix between segments Segment: spatiotemporal volume (3D), represented as a point in feature space (such as RGB histogram, local binary pattern histogram, or dense optical histogram). • Positive segment • Concept segment • Background segment • Negative segment Goal: classify the from in . Rank the elements in in decreasing order of a score, such that top-ranked Elements correspond to .

5. Interpretation framework • Baseline algorithms about segment annotation. • Kernel density estimation for Negative segments. • Intuition: the distribution of is similar to distribution of . • Construct a probability density operated on block C. • Rank the elements according to . • Negative Mining (MIN) • Intuition: distance from to the nearest > distance from to nearest . • Operated on block D.

6. CRANE • Each negative segment in penalizes nearby segments in . • Segments in should be those far from negatives. Penalty function

7. CRANE • Advantages Robust to noise. Parallelizable.

8. Experimental Results YouTube Objects datasets

9. Experimental Results • (a): Sucesses (b): Failures