1 / 104

Field studies

Field studies. User studies. Ubicomp: people use technology Must conduct user studies Also: Focus groups Ethnographic studies Heuristic evaulations Etc. User studies. Laboratory studies: Controlled environment Field (in-situ) studies Real world. Field studies.

laken
Download Presentation

Field studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Field studies

  2. User studies • Ubicomp: people use technology • Must conduct user studies • Also: • Focus groups • Ethnographic studies • Heuristic evaulations • Etc.

  3. User studies • Laboratory studies: • Controlled environment • Field (in-situ) studies • Real world

  4. Field studies • Appropriate for ubicomp: • Abundant data • Observe unexpected challenges • Understand impact on lives • Trade-off: • Loss of control • Significant time and effort

  5. Three common types • Current behavior • Proof of concept • Experience with prototype

  6. How to think about user studies? • Formulate hypotheses

  7. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

  8. What is a hypothesis? • Proposing an explanation • Theory or hypothesis? • “This is just a theory.” • Some theories we live by (“just” not justified): • Newton’s theory of motion • Einstein’s theory of relativity • Evolutionary theory

  9. Hypothesis • Must be tentative • Must predict

  10. Hypothesis • Some criteria of scientificity • Self-consistent • Grounded (fits bulk of relevant knowledge) • Accounts for empirical evidence • Empirically testable by objective procedures of science • General in some respect and to some extent

  11. On proposing hypotheses • Anomalous phenomena: • Strange and unfamiliar (Bermuda triangle) • Familiar yet not fully understood (cognitive load) • Is there already an explanation?

  12. Types of hypotheses • Incremental • Fundamental shift: • Ptolemy (c. 90 – c. 168): geocentric cosmology • Copernicus (1473 – 1543): heliocentric cosmology

  13. And then came… • Kepler (1571 – 1630): elliptical orbits

  14. Fundamental shift example • Ulcer: • Stress? Spicy food? • Bacteria.

  15. Types of proposed explanations • Causes • Correlation • Causal mechanisms • Underlying processes • Laws • Functions

  16. Proposing causal explanations • Studies show that using a cell phone while driving increases the probability of getting into an accident. Why is that so? • Pick up ringing phone • Dial number • See but don’t perceive

  17. Effects not always there • Cell phone + driving: • Usually no accident • Only one of the factors

  18. Remote and proximate causes • Cell phone + driving: • Attention shift → missed signal → accident • Remote cause → proximate cause → effect

  19. Correlation • A and B are correlated if: • A → B • B → A • C → A and C → B • A combination of (some of) the above • Coincidence • Correlation vs. causal relation: • Correlation doesn’t imply causal relation • Cannot determine cause direction (A → B or B → A)

  20. Correlation • Positive, negative • None found ≠ none exists • Causal link → correlation: • May provide initial evidence for causal link • Less explanatory value than facts about causal links

  21. Causal mechanisms • Mechanisms connecting remote causes and their effects. • E.g.: • Damaged artery in heart → clotting • Clotting → blocked artery • Blocked artery → heart attack • Aspirin inhibits clotting → lower risk of heart attack

  22. Underlying processes • Photoelectric effect

  23. Photoelectric effect • Einstein: 1921 Nobel Prize in Physics

  24. Laws • General regularities in nature • Universal: • F = ma • Non-universal: • Statistical laws

  25. Functions • What is the purpose of the phenomenon? FOR SALE A prime lot of serfs or SLAVES GYPSY (TZIGANY) Through an auction at noon at the St. Elias Monastery on 8 May 1852 consisting of 18 Men 10 Boys, 7 Women & 3 Girls in fine condition

  26. Functions • William Harvey (1578 – 1657): • Heart pumps blood through circulatory system • No modern instruments! • Experiments with a numberof animals: • Various fish, • Snail, • Pigeon, etc.

  27. Multiple methods together • Function → → causal mechanism → → underlying processes • National Ignition Facility(Dennis O’Brien @ UNH):Ignition with lasers → → Laser, target chamber → → Physics of nuclear fusion

  28. Multiple methods together • Law → underlying processes • Isaac Newton (1643 – 1727), second law of motion:F = ma → Graviton?

  29. Ockham’s razor • Crop circles: pranksters or aliens?

  30. Ockham’s razor • William of Ockham (c. 1288 – c. 1348) http://en.wikipedia.org/wiki/File:William_of_Ockham.png

  31. Do I have a hypothesis? • Yes. • Do you realize you do?

  32. How to think about user studies? • Formulate hypotheses

  33. Three common types • Current behavior • Proof of concept • Experience with prototype

  34. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

  35. Current behavior • Insights and inspiration: • State problem(s), goal(s) • Propose hypotheses • Relatively long

  36. Current behavior – example 1 • AJ Brush and KoriInkpen, “Yours, mine and ours?...” (pdf) (2005 movie inspiring title) • Home technology: users share, etc.

  37. Current behavior – example 2 • Schwetak Patel et al. “Farther Than You May Think…” (pdf) • Hypothesis: Mobile phone a proxy to user location.

  38. Three common types • Current behavior • Proof of concept • Experience with prototype

  39. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

  40. Proof of concept • Technological advance: • Produce results: prototype • Explain results: prototype • Relatively short

  41. Proof of concept – example 1 • J. Sherwani et al., “Speech vs. Touch-tone: Telephone Interfaces for Information Access by Low Literate Users” (pdf) (video) • Hypothesis: Speech better telephony interface than touch-tone for low literate users.

  42. Proof of concept – example 2 • John Krumm and Eric Horvitz, “Predestination:…” (pdf) • Hypothesis: Destinations from partial trajectories. • Train/test algorithm on GPS tracks from 169 people • Used pre-existing data: • Krumm and Horvitz, “The Microsoft Multiperson Location Survey” • Collecting original data a significant contribution • Leverage!

  43. Three common types • Current behavior • Proof of concept • Experience with prototype

  44. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

  45. Experience with prototype • Users’ interaction with technology: • Produce results: prototype • Explain results: prototype • Relatively long

  46. Prototype an example! • Others don’t care about: • Raw usage information • Usability problems • Intricate implementation details • Etc. • Generalize! • Scientific and good technical work

  47. Experience – example 1 • C. Neustaedter, et al., “A Digital Family Calendar in the Home:…” (pdf) (video) • Hypothesis: At-a-glance awareness, remote access are significant benefits. • 4 households, 4 weeks each • (Best Student Paper, Graphics Interface 2007)

  48. Experience – example 2 • Rafael Ballagas et al., “Gaming Tourism:…” (pdf) (video) • Hypothesis: Learning through a game. • 18 participants: 2 alone + 8 pairs (8 x 2 = 16)

  49. Study design • Who is the consumer? • Manager(s) • Industry, academic lab • Professor(s) • E.g. thesis committee • Researchers • E.g. advisor’s collaborators • Reviewers • For paper, proposal, thesis • Funding agency • Report on progress, proposal for funding • Public • Friends, family, alumni, potential students, donors, potential employers

  50. Study design • How can I explain this to a layperson? • What is key? What can be omitted? • How will I write this up? • Paper • Thesis • Report • Blog post • Start writing paper/thesis/report/blog post at the beginning of the study.

More Related