1 / 30

WP6 – Intelligent Interfaces

WP6 – Intelligent Interfaces. Enabling Access to Sound Archives through Integration, Enrichment and Retrieval. Introduction to Workpackage. Two fundamental objectives: Advanced research on intuitive human computer interface dedicated to audio and media resources interaction.

wilson
Download Presentation

WP6 – Intelligent Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WP6 – Intelligent Interfaces Enabling Access to Sound Archives through Integration, Enrichment and Retrieval

  2. Introduction to Workpackage Two fundamental objectives: • Advanced research on intuitive human computer interface dedicated to audio and media resources interaction. • Production of an operational and efficient intelligent interface.

  3. Introduction to Workpackage • T 6.1 Operational Interface design Based on previous user needs and partner’s background with existing tools, build an operational interface • T 6.2 Cognitive interface Develop new representation, manipulation, management tools using cognitive approach • T 6.3 Cross & multi-media interface Integrate WP3 retrieval system for multi-media and cross media experience

  4. Deliverables and Milestones • D6.1: Aligned Cross and Multimedia interface report (month 30) In progress: work done is continuously put in the deliverable. • Milestones: • M6.1 Mock GUI design devised (month 7) • M6.2 Audio intelligent interface (month 21) • M6.3 Multimedia Alignment interface (month 26) • M6.4 Cross-media and multimedia interface (month 29)

  5. Workpackage Progress Summary of progress: • Architecture • Interface: the approach • Interface: state of the art • Web Interface: mock-ups • Front-end application Interface • Realisation: method and work done

  6. Workpackage Progress Easaier architecture • Digital libraries review (Greenstone, DSpace, Fedora) • Web semantic solutions review (JeromeDL, SIMILE) • Architecture definition • Tests of IBM BOCA RDF storage solution • Integration of music ontology • Setup backend server (dual processor) at easaier.silogic.fr with RDF storage and web services capabilities

  7. Workpackage Progress Easaier architecture Defined during several meetings: • At Dublin with QMUL DIT Silogic • At Innsbruck with QMUL LFUI • At Toulouse with LFUI QMUL Silogic • And with input from other partners Backend

  8. Workpackage Progress Summary of progress: • Architecture • Interface: the approach • Interface: state of the art • Web Interface: mock-ups • Front-end application Interface • Realisation: method and work done

  9. Constraints Needs and first set of ideas Needs and coherent set of ideas, with the constraints. • User’s needs • Requirements • State of the art Good proposal for user. Mock-ups User tests Discussions with partners and users. Workpackage Progress Interface: the approach

  10. Workpackage Progress Summary of progress: • Architecture • Interface: the approach • Interface: state of the art • Web Interface: mock-ups • Front-end application Interface • Realisation: method and work done

  11. Workpackage Progress Interface: State of the art Audio visualisation – it’s important that users have the means to visualise information contained in an audio signal (frequency, periodicity, instrument segmentation..) Waveform Tonal landscape 3D View Time bar

  12. Workpackage Progress Interface: State of the art Query presentation– different methods of query appearance.In front of the quantity of information available in digital libraries, the approach proposes to the user to query a database is fundamental. Proximity search Dynamic Queries Query by example

  13. Workpackage Progress Interface: State of the art Query results presentation–it’s possible to optimise the display of a query result. User can find quickly a pertinent result, if the display is good. 3D network Self Organizing Map

  14. Workpackage Progress Interface: State of the art Archive browser– This enables the user to browse through the collection, an approach which is especially useful for discovering unknown pieces of music Classification browser Guided Tour

  15. Workpackage Progress Interface: State of the art Research on 3D : • 3D can bring intuitive perception of objects (virtual worlds… ) • Manage data complexity and volume (3D flow… )

  16. Workpackage Progress Interface: State of the art Research on 3D : • Auto-stereoscopic screens • New visualisation paradigm

  17. Workpackage Progress Summary of progress: • Architecture • Interface: the approach • Interface: state of the art • Web Interface: mock-ups • Front-end application Interface • Realisation: method and work done

  18. Workpackage Progress Web Interface: User needs • Search for media fitting a wide variety of metadata – The means to search across multiple and sometimes ambiguous selections of metadata. • Relevant near neighbour query-by-example searches – The possibility to an user to enter an audio sample, and the system produce ranked lists of related audio files, which may be similar in musical or timbal qualities. • Speech Retrieval tools. – A tool permitting a user to retrieve a audio speech based file. • A cross-media information retrieval system – Retrieve a different type of media as a related document • Enriched access and interaction tools – Different tools for enriching media retrieved elements, by using different filters, etc.

  19. Workpackage Progress Web Interface Main constraints: • Run in a web browser • Run on Windows, Linux

  20. Workpackage Progress Web Interface • Only one interface. • Form is organized in three main steps. • It’s possible to hide or show unused parts of fields

  21. Workpackage Progress Summary of progress: • Architecture • Interface: the approach • Interface: state of the art • Web Interface: mock-ups • Front-end application Interface • Realisation: method and work done

  22. Workpackage Progress Front-end application Interface • Start from Sonic Visualiser • Merge yellow areas • Lower interface complexity by hiding parts of the UI • Add search / browse capabilities • Add Real-time filter area

  23. Workpackage Progress Front-end application Interface • Display layers with their label • Show only visibility and audibility • Add buttons to change layers order • Hide layer parameters (accessible by double click)

  24. Workpackage Progress Summary of progress: • Architecture • Interface: the approach • Interface: state of the art • Web Interface: mock-ups • Front-end application Interface • Realisation: method and work done

  25. Workpackage Progress Realisation: method • Based on « spiral » model development cycle: • Analyse new needs • Specification and conception of functionalities • Realisation • Tests • Experimentations with end-users • Three versions already diffused to partners : alpha 1 & 2 & 4 for the advanced user application (version 3 internal).

  26. Workpackage Progress Realisation • Backend • RDF storage • SPARQL endpoint • Query Web service • Frontend • Client server connection • Search capabilities • Create layers associated to metadata retrieved on the server • New visualisation : Interval layer • Integrate DIT TSM filter

  27. Contributions and Connections with Other Workpackages • Input & output to: • WP2 Media semantics and Ontologies • WP3 Retrieval Systems • WP4 Sound object Representation • WP5 Enriched Access Tools • output to: • WP7 Evaluation and benchmarking

  28. Upcoming Work Plan Months 12-24 • Task 6.1: Operational interface • Develop Web client • Integrate partners modules (Real time filters, speech…) • Enrich visualisation paradigms (archive browsing, results presentation…) • Task 6.2: Cognitive interface • Research of new types of interfaces: navigation, presentation • Auto-adapt user interface to user profile • Task 6.3: Cross & Multi-media interface • Integrate retrieval systems • Add cross and multi-media visualisation

  29. Demonstration Web interface

  30. Demonstration Front-end client

More Related