Techniques and tools for field-based early-stage study and iteration of Ubicomp applications

1. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Scott Carter EECS Department University of California, Berkeley sacarter@cs.berkeley.edu Committee Jennifer Mankoff (chair), Anind Dey, John Canny, Peter Lyman 1

2. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter what are ubicomp apps? examples… social    proximity jabberwocky(paulos ’04)   location urban tapestry(lane ‘05)   activity resource queries    transit services   tour guide (abowd ’97) 2

3. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter what are ubicomp apps? examples… capture and access    field data        health (alivetech ’05)   reminder apps gaming    cysmn (blast theory ’01)   location,proximity-based 3

4. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp evaluation is hard (Carter ’05) 4

5. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp evaluation is hard rapid prototyping 5

6. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp evaluation is hard rapid prototyping scale 6

7. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp evaluation is hard rapid prototyping scale unobtrusiveness 7

8. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp evaluation is hard rapid prototyping scale unobtrusiveness data sparsity 8

9. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter thesis Leverage pre-existing infrastructures and the increasing power of mobile devices to address the challenges of early stage field evaluations of ubicomp applications. 9

10. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter why early stage and field? early stage >> potential to iterate >> could answer some research questions field >> realism especially important in ubicomp >> Sharp & Rehman ‘05: “Real-world deployment is the only way” >> Kjeldskov: might be able to recreate field in a lab 10

11. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter approach: WoZ researcher designs app behavior in momento and provides instructions for wizards wizards use momento to simulate app behavior 11

12. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter approach: report methods researcher designs experiment, configures momento or reporter researcher runs experiment, monitoring tool 12

13. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter limitations: when challenges do not apply data easy to get (no sparsity) infrastructure already available (no scale issues) target users early, early adopters (no unobtrusiveness) tools off-the-shelf (no prototyping) 13

14. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter most relevant related work WoZ Report methods ContextESM (Intille ’03) Topiary (Li ’04) rapid prototyping rapid prototyping scale scale unobtrusiveness unobtrusiveness data sparsity data sparsity other work other work SUEDE (Klemmer ‘00) iESP (Intel ’03) DART (Dow ‘05) Mobile probes (Hulkko, ‘04) Doorman (Makela ‘01) Crossweaver (Sinha ‘03) Active Capture (Chang ‘05) 14

15. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter outline ubicomp challenges case study momento report methods validation plans timeline summary 15

16. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp challenges case study: dr. blue social mobile game get points by collecting cures, healing the sick 16

17. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp challenges case study: dr. blue 17

18. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter ubicomp challenges case study: dr. blue 18

19. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter case study: dr. blue: rapid prototyping major difficulty: building a mobile application communication sensor display also difficult: game engine 19

20. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter case study: dr. blue: scale everyone needs a device and recording method 20

21. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter case study: dr. blue: unobtrusiveness new device and interface interruptions: evaluation and application behavior 21

22. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter case study: dr. blue: sparsity encounters may be limited 22

23. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter outline ubicomp challenges case study momento report methods validation plans timeline summary 23

24. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento WoZ simulation using SMS/MMS infrastructure 24

25. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento WoZ simulation using SMS/MMS infrastructure … and J2ME on the mobile device 25

26. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento input and output 26

27. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: building dr. blue 27

28. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: building dr. blue 28

29. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: building dr. blue 29

30. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: building dr. blue 30

31. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: building dr. blue 31

32. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: limitations SMS/MMS, wizard latency 24-hour wizards? synchronous and streaming data gui creation (damask – Lin ’02) 32

33. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter momento: summary scale rapid prototyping any device supporting sms/mms easy to change application behavior unobtrusiveness data sparsity familiar devices, but interruptions esm feedback on critical events participants integrate context more control wizards rules to take advantage of external services

34. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter outline ubicomp challenges case study momento report methods validation plans timeline summary 34

35. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter report methods diary study experience-sampling method (esm) 35

36. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter diary study issues unobtrusiveness neglect due to burden of study data sparsity generic responses no feedback scale data re-entry, transcribing 36

37. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter diary study technique (Carter & Mankoff ’05) capture annotate upload feedback elicitation 37

38. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter diary study technique: reporter 38

39. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter reporter scale rapid prototyping any digital capture device - no transcribing unobtrusiveness data sparsity familiar devices rapid feedback cycle no interruptions responses tied to captured artifacts feedback not in situ 39

40. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter esm: momento scheduled or context events automatic question formatting 40

41. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter esm: momento scale rapid prototyping many devices knowledge feedback to application behavior unobtrusiveness data sparsity familiar devices, but interruptions responses in situ participants redirect questions integrate context 41

42. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter outline ubicomp challenges case study momento report methods validation plans timeline summary 42

43. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter validation: diary study pipeline testbed: diary study at uc berkeley interviews with researchers compare to traditional diary studies (literature) success? unobtrusiveness fewer instances of neglect due to burden of study data sparsity more specific recall, fewer generic responses scale reduce time to data analysis (no data re-entry) 43

44. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter validation: momento esm study sensor data for awareness app location, activity data interviews with wizards, participants success? unobtrusiveness minimal neglect due to burden of study data sparsity useful feedback on critical events scale wizards able to manage momento queries and complete other work 44

45. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter validation: momento scale study implement dr. blue wide deployment (>~30) feedback on critical events, retainment success? rapid prototyping iteration time shorter than similar social mobile apps unobtrusiveness minimal neglect due to burden of study data sparsity useful feedback on critical events scale wizards able to manage momento queries and complete other work 45

46. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter contributions identification of major challenges in ubicomp evaluation: rapid prototyping, scale, unobtrusiveness, and data sparsity a tool (momento) that improves evaluations by simplifying prototyping, making them easier to scale, reducing obtrusiveness, and facilitating data collection a diary study technique that is easier to scale, less obtrusive, and yields better quality data than previous diary study techniques validation of momento and diary study technique 46

47. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter timeline 47

48. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter timeline 48

49. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter timeline: diary study 49

50. Techniques and tools for field-based early-stage study and iteration of Ubicomp applications Carter timeline: momento 50