Ethnography

1. Ethnography • Useful method studying people’s behavior and understandings • Can learn from anthropologists, sociologists, others who have extensive experience with this method • Also in IS214, Usability • IS272 – Qualitative Methods – addresses in more detail

2. Ethnography • “Properly done, provides detailed insight into the concepts and and premises that underlie what people do – but that they are often unaware of.” Forsythe • Methods • Participant observation • Observation • Audiotaped, videotaped • Interviews • Formal and informal • Analysis of records • Analytical perspective • Not beginning with theory but with observation • “grounded theory” • “Make the familiar strange”

3. Examples from our readings • Ling: observing people using cell phones in public spaces • “experimenting,” e.g., crowding them • Taylor & Harper: “ethnographically-informed” study of young people’s use of mobile phones • 18 hours of observation of phones and phone-related use in college cafeteria, hallway, and surrounding area >> field notes • 8 video-taped group interviews with 6 students, over 10 weeks and various locations

4. Presuppositions • Commitment to studying activities in natural settings • concern with understanding relation of particular activities to the constellation of activities and resources that characterize a setting • Detailed descriptions of lived experience • how people actually behave, not (just) their accounts • withhold judgment, recommendations, design • Members’ point of view • Use their categories, language • Your point of view affects what you see and understand

5. Ethnography vs. Ethnomethodology • Ethnography: method(s) and orientation • Ethnomethodology: analytic perspective • Study of a group’s/area’s common sense knowledge, procedures, considerations by which people make sense of and act on circumstances in which they find themselves • Parallels ethnobotany, ethnopsychology…

6. Impetus for Ethnography in HCI • To understand human-computer interaction, need to understand social and material contexts of interactions • Awareness that human intelligence socially created/achieved; can’t replicate in devices • Desire to support cooperative human activity (e.g., CSCW)

7. Central Premises • It is difficult for people to articulate tacit knowledge and understandings of familiar activities • So we observe them as well as talk to them • Participants act (toward technology) based on their own understandings and meanings • So we listen to them as well as observe them

8. Ethnographic Tools • Field notes • Make detailed notes on what is observed • Need to be done as soon as possible during/after observation • Separate interpretation from observation • Photos, video/audiotapes, & transcripts • Reusable record of exactly how people act, what they say • Repeated observation reveals unseen details • Precise wording used by participants may be revealing (e.g., Taylor & Harper)

9. Time • Collect data over a long enough period of time to question own assumptions, viewpoint • Each observer brings own experience, understandings, values, which may differ from the participants’ • Work with participants long enough to gain their trust • E.g., a series of personal failures may indicate systemic failure • Underlying issues, problems, practices, habits, assumptions may shape what you observe • Stay in a setting to see situation unfold over time • See situation through multiple viewpoints • Each participant has limited view, also

10. Ethnography and HCI • studies of work • where new technology might be intro’d but w/o explicit design agenda • studies of technology in use • situated use of specific technologies, classes of technology • participatory/work-oriented design • people who use/are affected involved in design – based on their understandings of their work

11. Ethnography and Systems Building • Gathering customer requirements: Understand their work, context, interactions – on site • Prototype evaluation: e.g., PARC work-oriented design project, put a working prototype in the workplace • Field evaluation: study use and integration of product/service on site

12. Ethnography and Studying Use/Behavior • Advantages: • Real world – observing technology-in-use • Settings, uses, users, conditions of use… • Can see things people wouldn’t think to report • Can ask questions at moment of interesting activity • Disadvantages: • Can only study a limited number of people, who are willing to have you around • Can you follow people around without affecting their behavior? • Teens and cell phones? • Do you have the time required to observe range of activity?

13. Misconceptions & ResponsesForsythe, “It’s Just a Matter of Common Sense” 1) Anyone can do ethnography – it’s just a matter of common sense. 2) Being insiders qualifies people to do ethnography in their own work setting. 3) Since ethnography does not involve preformulated study designs, it involves no systematic method at all – “anything goes.” 4) Doing fieldwork is just chatting with people and reporting what they say. 5) To find out what people do, just ask them! 6) Behavioral and organizational patterns exist “out there” in the world; observational research is just a matter of looking and listening to detect these patterns.

14. Responses • Anyone can do ethnography: Actually, ethnography runs counter to common sense, since it requires one to identify and problematize things that insiders take for granted. It takes a good deal of training and experience to learn to do this. It may also take courage on occasion, since insiders tend to experience their own assumptions as obvious truths. 2) Being insiders qualifies people to do ethnography: Ethnography usually works best when conducted by an outsider with considerable inside experience. The ethnographer’s job is not to replicate the insiders’ perspective but rather to elicit and analyze it through systematic comparison between inside and outside views - includes detecting tacit knowledge, something that by definition is generally invisible to insiders.

15. Responses (cont) 3) Ethnography involves no systematic method: anthropologists see ethnographic work as technical in nature and take seriously issues of methodological appropriateness, procedure, and validity. Proper ethnography involves systematicmethod and epistemological discipline. 4) Doing fieldwork is just chatting with people: Doing fieldwork certainly involves talking to people, but this is no more the entire task than system buildingis “just typing” or medical diagnosis is “just talking to patients.” The important point is what one is doing when typing or talking. Competent fieldworkers do not take what people say at face value; they treat people’s views as data, not results. The job of the social scientist is to understand and analyze what people say.

16. Responses (cont) 5) Just ask them: Ethnography does entail eliciting people’s understandings of their own and others’ behavior, but only the most naive of fieldworkers would treat such understandings as reliable data about systematic behavioral patterns. Anthropologists know from our observational tradition hat people’s verbal representations of their own behavior are often partial and sometimes incorrect. In other words, it is imperative to watch people engaged in activity as well as to ask them about it. 6) Behavioral and organizational patterns exist “out there” in the world: Observational research is sometimes perceived by others as just a matter of looking to see what is “out there”; one need only look and listen…This common misconception fails to grasp is the selectivity and interpretation that go into the process of gathering careful ethnographic data, writing useful fieldnotes and analyzing the data in an appropriate and systematic way.

17. Difficulties with Ethnography • Harder to do well than it appears • High resources demands • Human resources – time and expertise • Calendar time • Lots of information to analyze • Difficult to translate observations and understandings for others • How to link to design? • How to use to develop designs for more general use, other than this setting?

18. But: • Useful as an orientation, set of principles • Important reminder to stay grounded in the users’ actual experience and understandings • Useful reminder to observe and listen

19. Rapid Ethnography – in general • Team of researchers – divide up, share observations, interact with one another • Triangulation: multiple data collection methods • But should be doing this anyway • Iterative data collection and analysis • Ditto

20. “Quick and dirty” Ethnography • Focused interviews • Unattended video • “Interactive feature conceptualization” • Most often: short time, focused, fast in and out

21. Key Elements of Rapid Ethnography (Millen) • Narrow the focus of field research before entering field. • Important activities • Key informants • Use multiple interactive observation techniques, looking for exceptional and useful user behavior • Use collaborative and computerized iterative data analysis methods

22. How rapid ethnography (Millen) diverges from traditional ethnography • “Objectivity” • Speed • Focus • “Exceptional” occurrences • Interpretation

23. Downside of Rapid Ethnography • Too focused • Too narrow a view • Find what you expect to find • Too few informants to get a broad view, find the people you really need • Not enough understanding of the situation to know • What’s important • When to collect data • Whom to talk with • How participants understand situation • Not enough time • for exceptions to surface or patterns to appear • to build trust • to shift own thinking to understand nuances of situation • to understand informants’ relationships to situation • Not skeptical enough – or too skeptical

24. Other Ways to Adapt Ethnography to Rapid Development Cycles • Uncouple research schedule from product development cycle • On-going data collection to feed into design when and as needed • Pair ethnography with short-term methods (like focus groups) • Look ahead several iterations of product design

25. Some Issues in Ethnography • Your relationship with site, participants • Jeopardizing your future relationship, access? • Identification with the people you talk with • Reconciling multiple points of view • Confidentiality and trust • What happens when you know something they don’t know? • When to stop? How much is enough? • How much control to give participants over your report? • What to do when the greatest problems/needs are supposedly outside the scope of your study?