1 / 70

Lecture #1 0 Multimodal Interfaces and Accessibility

Lecture #1 0 Multimodal Interfaces and Accessibility. *39TUR. Multimodal User Interfaces. Attempts to use human communication skills Provide user with multiple modalities May be simultaneous or not Multiple styles of interaction.

garry
Download Presentation

Lecture #1 0 Multimodal Interfaces and Accessibility

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lecture #10Multimodal Interfaces and Accessibility *39TUR

  2. Multimodal User Interfaces • Attempts to use human communication skills • Provide user with multiple modalities • May be simultaneous or not • Multiple styles of interaction This presentation contains materials byGeorge G. Robertson / Microsoft Research and Adam Sporka / University of Trento TUR 2012

  3. Why Multimodal User Interfaces • Use more of user’s senses • Not just vision • Sound • Tactile feedback • Users perceive multiple things at once • e.g., vision and sound TUR 2012

  4. Modalities • Visual • Primary in the UI • Acoustic • Often used • Tactile • Complementary, but still important • Used by the vision-impaired people (Braille lines) TUR 2012

  5. Modalities • Olfactory (smell), gustatory (taste) • Experiments • “When you smell burning plastic, you know the machine has failed.” • Food simulator • http://www.siggraph.org/s2003/conference/etech/food.html TUR 2012

  6. Modality • Many styles exist • Command interface • Natural Language • Direct manipulation (WIMP and non-WIMP) • Conversational (with an interface agent) • Collaborative • Styles vs. I/O devices • Mixed styles produce multimodal UI • Direct manipulation and conversational agent TUR 2012

  7. Examples • Put That There (1970s, early 1980s) • Speech and gestures used simultaneously • http://www.youtube.com/watch?v=sC5Zg0fU2e8 • Typical GPS devices in cars • Visual modality (screen) • Acoustic modality (speech output) • Computer games TUR 2012

  8. Multimodal versus Multimedia • Multimedia is about media channels • Text, graphics, animation, video: all visual media • Multimodal is about sensory modalities • Visual, auditory, tactile, … TUR 2012

  9. Modalities PERFORMANCE ARTICULATION  TYPING CONTROLLER POINTING  CONTROLLER SPEECH RECOGNITION  H C  HEARING TTS  VISION GUI  VISION OBSERVATION PRINT PRESENTATION TUR 2012

  10. People with Disabilities • Politically correct terminology evolves over time • Handicapped • The blind people • Vision-impaired, motor-impaired • People with Special Needs • Differently able • Mark Haddon (2003)The Curious Incident of the Dog in the Night-time • Can be permanent, can be temporary TUR 2012

  11. Accessibility • Usability • “Users know how to use the tool” • Accessibility • “Users possess senses and dexterity to operate the tool” • Being able to be accessed by as many people as possible TUR 2012

  12. Vision-Impaired People  TYPING CONTROLLER POINTING CONTROLLER SPEECH RECOGNITION  H C  HEARING TTS VISION GUI  BRAILLE TOUCH TUR 2012

  13. Motor-Impaired People TYPING CONTROLLER POINTING CONTROLLER SPEECH RECOGNITION  H C  HEARING TTS  VISION GUI  VISION PRINT TUR 2012

  14. Assistive Technology • An umbrella term for various techniques and devices that compensate the lack of dexterity and/or senses. • Various approaches • Screen readers • Speech recognition • Eye trackers • Head tracers TUR 2012

  15. Accessible technology Accessible technology is technology that is flexible and adjustable to an individual’s visual, mobility, hearing, language, and learning needs It enables computer access for people with impairments – regardless of their functional abilities Accessible technology solutions enable people of all abilities to realize their potential TUR 2012

  16. Modality Substitution  SPEECH HEARING H H  HEARING SPEECH Environment / Condition (Danninger et al., 2007) TUR 2012

  17. Modality Substitution (Danninger et al., 2007) TUR 2012

  18. Jan Volný (2011)

  19. Modality Conversion HEARING H  C  GUI VISION (Kildal et al., 2007) TUR 2012

  20. User Interfaces for the Elderly

  21. Condition • Reduced • Motor skills • Perception • Vision, hearing, touch, mobility • Speed • Memory • Other needs • Technology experience is varied (How many grandmothers use email? mothers?) • Uninformed on how technology could help them • Practice skills (hand-eye, problem solving, etc.) • Touch screens, larger fonts, louder sounds TUR 2012

  22. Usability for Senior Citizens • Provide large targets • Reduce the number of clicks • Do not require double-clicks • Do not use pull-down menus • Do not have a deep page hierarchy • Concentrate important information at page top • Avoid the need to scroll • Place most links in bulleted, not tightly clustered list • Clearly differentiated visited/non-visited links TUR 2012

  23. Usability for Senior Citizens • Use few colors, avoiding blue and green tones • For text to be read or scanned, use: • 12- to 14-point sans serif font (Helvetica, Arial)? • Black text on white background • Left alignment • Increased spacing (leading) between lines • Sentence style mixed-case letters • Appropriate large headings in a 14- to 16-point sans serif font TUR 2012

  24. Vision Impairments

  25. Colorblindness • Normal vision • Perception of three wave-lengths, corresponding to red, green, and blue • Perception of a color  “RGB ratio” • Trichromacy Stockman, MacLeod & Johnson (1993) TUR 2012

  26. Flatla & Gutwin (2012) http://80.dx.doi.org.dialog.cvut.cz/10.1145/2384916.2384946

  27. Colorblindness • Colorblindness • Inability to perceive all three “channels” • Males: 8 %, Females: <1 % • Types • Monochromacy • “Grayscale vision” • Very rare • Dichromacy • One type of cones is absent • Protanopia (R), Deteranopia (G), Tritanopia (B) • Anomalous trichromacy • One type of cones is anomalous TUR 2012

  28. Colorblindness • User Interface • Visited / unvisited links • Alert messages do not stand out • Everyday life problems • “Is the fruit ripe?” • “Is the meat rare or well-done?” TUR 2012

  29. Simulation Images made using http://www.vischeck.com/ TUR 2012

  30. Simulation TUR 2012

  31. Simulation TUR 2012

  32. Personalized Simulation Original Personalized Generic Flatla & Gutwin (2012) http://80.dx.doi.org.dialog.cvut.cz/10.1145/2384916.2384946 TUR 2012

  33. Personalized Simulation Flatla & Gutwin (2012) http://80.dx.doi.org.dialog.cvut.cz/10.1145/2384916.2384946 TUR 2012

  34. Personalized Simulation Flatla & Gutwin (2012) http://80.dx.doi.org.dialog.cvut.cz/10.1145/2384916.2384946 TUR 2012

  35. Compensation • Traffic lights • Workaround: Determining the R/G light by the position • Replacement of green with blue (Japan) • Shape coding (Canada) • Visualization • The color should not be the singlecarrier of information • Maximize the contrast • Use some texture, use captions • Allow the users set up the color scheme • Daltonization http://raynefeather.deviantart.com/art/Rainbow-Dash-quot-Grrr-quot-Face-272751637 TUR 2012

  36. Daltonization • Maximize the differences between colors TUR 2012

  37. Blindness • Severity • Normal vision • Vision corrected to normal • Loss of acuity • Counting Fingers, Hand Motion, Light Perception • No Light Perception TUR 2012

  38. Various Anomalies • Tunnel vision • Loss of peripheral vision • Can count coins, can’t cross the street • Macular degeneration • Loss of vision in the center of the visual field • Akinetopia • Inability to perceive moving objects • Hemispatial neglect • Inability to perceive objects on one side TUR 2012

  39. Impacts on the UI • Reduced ability to perceive feedback from the machine • Reduced ability to perceive the physical device • Layout of the buttons • Compensation • Modality substitution • Simplification of the machine TUR 2012

  40. Motor-impaired Users

  41. Causes • Congenital disorders • Injury • Spinal cord • Loss of limbs • Other medical conditions • Cerebral palsy • Arthritis TUR 2012

  42. Consequences • Poor muscle control • Difficulty sensing, grasping, reaching • Difficulty making fine motor movements with fingers • Difficulty doing compound manipulations • Inability to exert much force on controls TUR 2012

  43. Consequences • Limited movement of arms • No function of arms • Limited or no articulation in fingers • Standard devices present a challenge • Keyboard, Mouse, Joystick • No or very slow use TUR 2012

  44. Effects on the UI • Some people don’t type fast • E.g. Friedreich’s Ataxia patients – 15 to 30 characters per minute • This affects any text input, including chat and search • Some people don’t use mouse • Limited use of keyboard or use of a dedicated hardware instead. • This affects the navigation in the software • Some people can’t use touch screens TUR 2012

  45. Cognitive Disabilities

  46. Cognitive disabilities TUR 2012 Memory Problem-solving Attention Reading, linguistic, and verbal comprehension Math comprehension Visual comprehension (46)

  47. Cognitive disabilities http://webaim.org/articles/cognitive/ What is being said in this phrase? Tob eornot obe TUR 2012 (47)

  48. Accessibility Standards and Legislation

  49. Motivation • Everybody is allowed to receive information from public institutions • Everybody is required at some point to communicated with public institutions • Bureaus, offices, ministries, etc. • People with disabilities need other people to access these services • Unless, the services are accessible • Accessibility is mandatory for public services in many countries worldwide TUR 2012

  50. Motivation • Accessibility • Transportation • Buildings • Software • Websites TUR 2012

More Related