1 / 84

Steerable Interfaces for Interactive Environments

Institut National de Recherche en Informatique et Automatique. Stanislaw Borkowski thesis director: James L. Crowley Jury:. Steerable Interfaces for Interactive Environments. Andreas Butz (UM), Jo ë lle Coutaz (UJF), Alex Pentland (MIT), Pierre Wellner (IDIAP). INRIA Rh ô ne-Alpes

mallory-slater
Download Presentation

Steerable Interfaces for Interactive Environments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Institut National de Recherche en Informatique et Automatique Stanislaw Borkowski thesis director: James L. Crowley Jury: Steerable Interfaces for Interactive Environments Andreas Butz (UM), Joëlle Coutaz (UJF), Alex Pentland (MIT), Pierre Wellner (IDIAP) INRIA Rhône-Alpes June 26, 2006

  2. What is a user interface? • User Interface (UI): aggregate of physical entities or information bound to these entities A

  3. Mobile UI’s • Steerable UI: • can be relocated in space • position is mediated by the computer system • Portable UI: • can be relocated • position is directly controlled through physical contact Mobile UIs Steerable UIs Portable UIs A

  4. Mobility in current IT • Steerable interfaces • Conventional GUI (steerable output) • X11 session teleporting [Richardson93] • Portable interfaces • Wearable computers • Cell phones • Personal Digital Assistants • Laptops • ….

  5. Mobility in ambient computing • Multiple displays embedded in the environment • Large size displays • Mobile interaction resources, both portable and steerable [Arias00] [Streitz99] [Pinhanez01]

  6. Why steerable? • Flexibility in resources usage • New forms of Human-computer interaction • New forms of Human-Human interaction

  7. Current situation – summary Problem: • Need for steerable UIs • No predictive models Solution: • Provide enabling technology • Explore interaction techniques • Evaluate the value of steerable UIs

  8. Outline • Mobility in IT • Steerable UIs • Mobile projected UI • Mobile UIs for collaborative work • Conclusions

  9. State of the art • EasyLiving [Brumitt00] • Tic-Tac-Toe [Pinhanez05]

  10. State of the art – limitations • UI is observable at standstill • Limited spatial controllability • Only predefined locations • Planar surfaces only Requirements for steerable UIs: • Continuous observability and controllability

  11. Outline • Mobility in IT • Steerable UIs • Mobile projected UI • Prototype implementation[in collaboration with J. Letessier] • Evaluation – latency estimation • Mobile UIs for collaborative work • Conclusions

  12. The Steerable Camera Projector(2002) Other steerable projection systems: • The Everywhere Display (IBM 2000) • Fluid Beam (Fluidum consortium 2002) • SCP from Karlsruhe (UKA 2004)

  13. Steerable display (2002)

  14. User-centric approach End-users: • Latency limits < 50ms • Easy setup, no maintenance • Reliability / predictability Developers: • Abstraction: be relevant • Isolation: allow integration • Contract: offer quality of service

  15. Pragmatic approach • Black-box services • BIP(Basic Interconnection Protocol) • BIP implementation ≈ SOA middleware • service/service and service/application communication • service discovery (standards-based)

  16. Interactive system SCP software Human and Environment Interaction events Application Display orders

  17. Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller A

  18. Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller

  19. Screen Video projector Source image Projection on arbitrary oriented planar surfaces Light source User’s perception

  20. Screen Video projector Image to project Source image Projection on arbitrary oriented planar surfaces Light source SCPdisplay User’s perception

  21. Projection on arbitrary oriented planar surfaces User’s view Image to project

  22. Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller

  23. 3 Y 2 1 Q Sensor-centric environment model 2 3 1

  24. Display surface detection Screen

  25. The Portable Display Surface

  26. Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller

  27. Interactive widgets projected on a portable display surface

  28. Luminance-based button widget

  29. Touch detection • Locate widget in the camera image • Estimate occlusion • Update widget state

  30. Robustness to clutter

  31. Assembling occlusion detectors

  32. Striplet – the occlusion detector x y

  33. Striplet-based SPOD SPOD – Simple-Pattern Occlusion Detector

  34. Striplet-based button

  35. SPOD-based calculator Accelerated video

  36. Outline • Mobility in IT • Steerable UIs • Mobile projected UI • Prototype implementation • Evaluation – latency estimation • Mobile UIs for collaborative work • Conclusions

  37. PCI A/D converter Frame Grabber Graphic Card OpenGl render Latency estimation CPU Imalab shell Image processing PDS

  38. PCI A/D converter Frame Grabber Graphic Card OpenGl render Latency estimation Video sequence capture CPU Imalab shell Projection of the bar Image processing Plastic bar Fan Regulated power supply

  39. PCI A/D converter PCI A/D converter Frame Grabber Frame Grabber ~17ms CPU Imalab shell Image processing ~70ms Graphic Card OpenGl render CPU Imalab shell ~32ms Graphic Card Latency estimation – results A + up to 51ms!!!

  40. Human and Environment Interactive system Interaction detector Frame grabber Application SCPdisplay SCPcalibrator SCPcontroller

  41. Outline • Mobility in IT • Steerable UIs • Mobile projected UIs • Mobile UIs for collaborative work • ContAct application • User study – comparison of different take-over techniques • Conclusions

  42. ContAct – a system for authoring presentations Collaboration through interface mobility

  43. ContAct application setup • Wide angle camera • Tabletop camera • Steerable Camera Projector • Portable Display Surface

  44. ContAct application GUI

  45. Outline • Mobility in IT • Steerable interface prototype • Mobile UIs for collaborative work • ContAct application • Taking control: a comparative user study [in collaboration with J. Maisonnasse and J. Letessier] • Conclusions

  46. Evaluation of techniques for taking control Objectives: • Determine the preferred control taking technique • Evaluate the impact on the task completion performance • Evaluate user acceptance of steerable interfaces

  47. Experimental setup Hardware: • Steerable Camera Projector • Microphone headsets • Portable Display Surface GUI Software: • Speech detector [D. Vaufreydaz] • Conversation modeling [J. Maisonnaisse] • Finger tracking [J. Letessier] • PDS tracking • Drawing application Users

  48. The User Interface

  49. The task Collaborative reconstruction of a graph

  50. The task Collaborative reconstruction of a graph User 1 User 3 User 2

More Related