Physical User Interfaces What they are and how to build them Saul Greenberg University of Calgary

1. Physical User InterfacesWhat they are and how to build themSaul GreenbergUniversity of Calgary

2. Physical User Interfaces • Special purpose computer-controlled devices that can be situated in a real-world setting. • typically designed for particular contexts and uses.

3. Physical User Interfaces • Foundations for • Ubiquitous Computing • Tangible Media • Context aware computing • Styles of use • foreground interaction • ambient displays • information appliance • collaborative interaction • physical controls

4. Ubiquitous Computing • A less-traveled path I call the invisible; its highest ideal is to make a computer so imbedded, so fitting, so natural, that we use it without even thinking about it. • Provide hundreds of wireless computing devices perperson per office, of all scales …It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere. Mark WeiserXerox PARC Source: Mark Weiser’s UbiqCom web site

5. Ubiquitous Computing • Invisible Everywhere Computing • invisible: • tiny, embedded, attachable… • everywhere: • wireless, dynamically configurable, remote access, adapting… • aka Pervasive Computing Mark WeiserXerox PARC Source: Mark Weiser’s UbiqCom web site

6. Makes use of simple shared context space time proximity affordances Participation in the context is physical is out here with us is in many small and large places, including trivial ones Ubicomp is Situated Computing Source: Mark Weiser’s UbiqCom web site

7. Ubicomp Technology Trends Source: Mark Weiser’s UbiqCom web site

8. Ubicomp Technology Trends • Processors: • cheap, small, dedicated, microprocessors • Low Power • small batteries, solar (?) • Wireless • 802.11, Bluetooth, infrared, mobile telephony, … • Displays • very small (inches) to very large (walls) • Peripherals • sensors, actuators, motors, … • Run-time systems • Linux on a chip, Windows CE, downloadable executables… Source: Mark Weiser’s UbiqCom web site

9. ParcTab • Mobile hardware • infrared • room-sized cells • location information • A portable GUI device • small case with belt clip, ½ size of PDAs • touch sensitive 128x64 pixels display • 3 finger-operated mechanical buttons (chorded) • piezo-electric speaker • low power needs (~ 1 week between charges ) • can be used in either hand by rotating display Source: Mark Weiser’s UbiqCom web site

10. Tangible User Interfaces • From ‘painted bits’ to ‘tangible bits’ • genre change • give physical form to digital information • physical objects, surfaces, and spaces become tangible embodiments of digital information. • seamlessly couple the dual worlds of bits and atoms • input: grasp and manipulate • output: change of physical properties of object Hiroshi IshiiMIT Source: Tangible Media Group web site

11. Context-Aware Computing • Context as information • … characterizes a situation of a person, place or object relevant to the interaction between a user and an application • location • identity • state and activites of people, groups • state of computational and physical objects • Context-aware computing • uses contextual information to • selectively present information and services • automatically execute a service • attach context information for later retrieval Source: Dey, Abowd and Salber, HCI Journal 2001

12. Styles of useForeground interactions • Conscious intentional interactions • graspable objects • augmented surfaces • link between physical and virtual actions • exploit human senses of touch and kinesthesia.

13. Interaction One opens and closes bottles to control digital contents e.g., opening it tells a story Music bottles movement and uncorking of the bottles controls the different sound tracks and the patterns of colored light rear-projected onto the table’s translucent surface. MusicalBottles - controlling sound, light Source: Tangible Media Group web site

14. MusicalBottles - controlling sound, light Ishii, MIT Source: Tangible Media Group web site

15. IAMASCOPE

16. Wooden Mirror - wooden pixels, image Daniel Rozin, NYU Ishii, MIT

17. Wooden Mirror - wooden pixels, image Ishii, MIT Daniel Rozin, NYU

18. When triangles connect together, they trigger digital events. These events influence the progress of a non-linear story, or allow users to organize media elements in order to create their own story space. Triangles – building a digital story Extracted from Tangible Media Group web site

19. Triangles – building a digital story Extracted from Tangible Media Group web site

20. Ambient displays • Background information • communicate digitally-mediated senses of activity and presence at the periphery of human awareness. • ambient light sound, airflow, water movement, object motion… • peripheral displays

21. Extracted from Mark Weiser’s UbiqCom web site

22. Extracted from Mark Weiser’s UbiqCom web site

23. Pinwheels - ambient digital wind Extracted from Tangible Media Group web site

24. Pinwheels - ambient digital wind Extracted from Tangible Media Group web site

25. Information Perculator • bubbles of digital patterns Heiner, Hudson & Tanaka

26. Information Perculator • bubbles of digital patterns Heiner, Hudson & Tanaka

27. light reflection from water onto ceiling Ambient Light Display

28. Ambient Room ACM CHI’98, Tangible Media Group

29. Small, physical devices worn to display information to a person in a subtle, persistent, and private manner. Ambient information is displayed solely through tactile modalities such as heating and cooling, movement and vibration, and change of shape. Personal Ambient Display Extracted from Tangible Media Group web site

30. Physical controls • Physically-mediated computer controlled interactivity

31. Incoming voice messages are physically instantiated as marbles. The user can grasp the message (marble) and drop it into an indentation in the machine to play the message. The user can also place the marble onto an augmented telephone, thus dialing the caller automatically. Marble Answering Machine Durrell Bishop

32. Computational tags track the usage of physical objects. TouchCounters sense activity through magnetic, acceleration, and infrared sensors, and indicate their status on bright LED displays. TouchCounters can be networked to a web server that generates use histograms for each object. TouchCounters Extracted from Tangible Media Group web site

33. TouchCounters Extracted from Tangible Media Group web site

34. Tagged Objects ACM CHI’99, Xerox Parc

35. mediaBlocks Extracted from Tangible Media Group web site

36. A system for tracking the positions and states of multiple objects wirelessly on a flat surface. Objects can be equipped with various controls -- dials or buttons -- which can be monitored in real-time. When coupled with a projector, the system can display information about the objects on or near the objects themselves. SenseTable Extracted from Tangible Media Group web site

37. A vectorizing digital tape measure for digitizing field measurements, and visualizing the volume of the resulting vectors with computer graphics. Using embedded orientation-sensing hardware, it captures relevant vectors on each linear measurements and transmits this data wirelessly to a remote computer in real-time. HandScape Extracted from Tangible Media Group web site

38. Manipulative User Interfaces

39. Customizable Physical Interfaces ACM UIST 2002, Greenberg & Boyle

40. Collaborative interactions • Physical objects connecting collaborators • collaborative interaction • awareness Adapted from Tangible Media Group web site

41. features a "reactive table" that incorporates sensing, sound, and projection technologies. Projectors display patterns of light and shadow on the table; bouncing balls leave images of rippling water; and the rhythm of play drives accompanying music and visuals. PingPong Plus Extracted from Tangible Media Group web site

42. PingPong Plus Extracted from Tangible Media Group web site

43. Force-feedback technology is employed to create the illusion that people, separated by distance, are interacting with a shared physical object. When one of the rollers is rotated, the corresponding roller on the other distant object rotates in the same way. Two people separated by distance can then play… InTouch– collaborative haptics Extracted from Tangible Media Group web site

44. InTouch– collaborative haptics Extracted from Tangible Media Group web site

45. …two cold steel benches located in different cities. When a person sits on one of these benches, a corresponding position on the other bench warms, and a bi-directional sound channel is opened. At the other location, after feeling the bench for "body heat," another person can decide to make contact by sitting near the warmth. Initially the sound channel is distorted, but as the second party lingers, the audio channel clears. --summarized by Ishii and Ullmer Bench Anthony Dunne and Fiona Raby at the RCA

46. Proximity detector ultrasonic sensor Hydra unitvideo, camera, speakers, microphone Rotating figurine servo motor Tippable figurine light sensors Physical but Digital Surrogates

47. Physical but Digital Surrogates

48. Privacy preserving media space

49. Computer-augmented room elements like doors, walls, furniture (e.g. tables and chairs) with integrated information and communication technology. Roomware i-land From the GMD Darmstadt web site on I-Land

50. Roomware i-land • Dynawall From the GMD Darmstadt web site on I-Land