1 / 21

Recent Advances in ViPER

David Mihalcik David Doermann Charles Lin. Recent Advances in ViPER. What is ViPER?. A tool for evaluating video understanding algorithms. Includes: An Annotation Tool For labeling ground truth and browsing results. A Comparison Tool

dea
Download Presentation

Recent Advances in ViPER

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. David Mihalcik David Doermann Charles Lin Recent Advances in ViPER

  2. What is ViPER? • A tool for evaluating video understanding algorithms. • Includes: • An Annotation Tool • For labeling ground truth and browsing results. • A Comparison Tool • For evaluating result data with respect to ground truth.

  3. What is the Problem? • Lots of people, here and elsewhere, are working on video processing algorithms for information extraction, etc. • Evaluating performance of the algorithms requires a lot of work, usually with tools developed by the algorithm designer. • Is your solution any good? Prove it.

  4. How to evaluate the algorithm? • What is the problem? • ViPER focuses on evaluating solutions to detection and tracking problems; these determine if or where in the video some entity or event appears. • Evaluation • Comparison of the result data set against a truth data set. • Truth, metrics, and rules for comparison are task dependent.

  5. Goal of the ViPER Project • To make evaluation of video algorithms simple, repeatable, and ubiquitous. • As ground truth is required for evaluation, annotation must be made simple, as well. • Avoid tedium. • Avoid frustration. • Support expert usage.

  6. Ground Truth Annotation • ViPER-GT supports annotation of temporally qualified spatial and nominal data on video files and still images. • Go from the simple per-frame or shot annotations to detailed spatial markup. • You can quickly indicate which frames contain people. • Then, you add how many people per frame. • With a lot of time and money, you can put boxes around them.

  7. ViPER-GT: Video Ground Truth Annotation Tool

  8. For Example: Person Tracking • How well does an algorithm find and track humans moving through a video? • To evaluate detection: • Truth must indicate which frames contain the person. • To evaluate tracking: • Truth must contain spatial information, indicating where a person may be found.

  9. Example of Annotation: Person Detection

  10. Example of Annotation: Person Tracking

  11. The ViPER Data Model • Similar to a relational database: • Tables are Descriptor Definitions. • Columns are Attributes. • Rows are Descriptor Instances. • Most descriptors are OBJECT descriptors: • Attributes are temporally qualified. • Static OBJECTS have a frame range, but their attributes are not temporally qualified. Useful for events, etc.

  12. The ViPER File Format and API • Uses XML. • I won’t go into it here. • There is a Java API.

  13. Related Work • VideoAnnEx • OntoLog • PhotoStuff • Informedia

  14. VideoAnnEx • IBM’s MPEG-7 annotation tool. • Cool Features: • Cut detection makes it easy to add per-shot markup. • Supports MPEG-7. • Annoyances: • Not very good for spatial attributes. • Commercial software. Not as extensible as ViPER.

  15. OntoLog • Jon Heggland’s Tool for Temporal Markup with Ontologies • Advantages: • Good support for key bindings and playback. • Data model supports inheritance. • Annoyances: • No spatial data support.

  16. PhotoStuff • MINDSWAP’s Tool for Adding Semantic Web Markup to Images • Cool Features: • Semantic Markup! • Spatial Data! • Annoyances: • Buggy and beta. • No support for video.

  17. Informedia • CMU’s tool for browsing video libraries • Cool Features: • Advanced browsing functionality. • But… • Focus on video library, not annotation. • Not available for download, from what I can tell. • Not terrifically extensible. • See also: Silver and Malach

  18. Extending the Interface • ViPER provides a lot of functionality, but is very general. • It may be appropriate to extend viper-gt to better support marking up a different type of annotation.

  19. Example Extension: Adding Text Zones • Adds a toolbar that allows typed bounding boxes. • Instead of having to click create, auto-creates a new box.

  20. Architecture of ViPER-GT • Application Launcher • Loads a set of javabeans from an RDF model. • Allows modifying menus, i18n, etc. • Is a bit of a pain to handle 'menu change' events, like for most recently used menu. • Viper View Mediator • Javabean container for ViPER API. • Adds 'user interaction' methods to keep track of things not in API (focus, filters).

  21. Continual Improvement SourceForge web site: http://viper-toolkit.sf.net/ Mail suggestions/comments to: viper@cfar.umd.edu

More Related