Characterizing Audience for Informational Website Design: A Case Study

1. Characterizing Audience for Informational Website Design: A Case Study Jennifer Turns, Ph.D. Assistant Professor, Technical Communication Faculty Affiliate Program for Educational Transformation through Technology (PETTT) Center for Engineering Learning and Teaching (CELT) Acknowledgements: This work has been supported by the Program for Educational Transformation through Technology (PETTT). Many people have contributed to this work including Scott Macklin, Tracey Wagner, Aaron Louie, Brett Shelton, Kristina Liu, Alice Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.

2. Overarching UCD Questions • How do design teams incorporate information about users into their design process? • When characterizing users/audience? • Which dimensions are most significant? • Which methods under which circumstances? • How will the insights inform design? • What are realistic expectations?

3. Informational Website Design Some of my projects in this domain… • Arthritis Source: Medical information on the web • Teaching challenges of engineering faculty: Information for educators • Also • Legal information online • Information about architecture building methods

4. Today’s Goal Use a case study of audience analysis for website design to reflect on decisions in audience analysis • Describe Arthritis Source, informational website • Describe our approach to audience analysis • Present sample of results from our audience analysis, and their impacts on the design • Discuss moving beyond the case study

5. Arthritis Source • Developed in 1995 by Dr. Frederick Matsen • Focus on arthritis • Authorized information • User-centered information • Research test bed

6. Arthritis Source Content • Articles as basic metaphor • Content is organized into articles • Examples: Rheumatoid Arthritis, Rotator Cuff Surgery, Pregnancy • Templates underlie articles: • The template for each article is the set of questions answered in each article • All content is based loosely on five templates: conditions, surgery, treatment, living with, medication • Templates are informed by research on user questions • Content is dynamically generated

7. Arthritis Source Authoring • Authors • Volunteer • Are subject matter experts (e.g., doctors) • Are not trained as technical communicators • Create content for entire template or specific question • Can create content online or in Word • Authoring is actually distributed • Team creates templates based on user information • “Authors” create content • Administrator edits

8. Multiple Access Paths • Multiple Access Paths • Navigation comes from article templates • Spotlighted content • Question-based search

9. Embedded, Ongoing Evaluation • Online survey • “Tell us about yourself” • Quick polls • “How useful is this article” • Online quizzes • “What do you know…” • Online research studies • “Exploring use over time”

10. Community includes Users Technical Communication Professionals Domain Experts (i.e., doctors) Learning Scientists Developers Administrator Share responsibility for Determining scope Creating content Evaluation quality Arthritis Source Community

11. Arthritis Source Timeline • Dynamic Content • Template-based content Embedded evaluations • Search EngineOptimization Question-based Navigation Creation Design ‘01 ‘00 ‘02 1995 Audience Analysis • Audience Analysis • Designing begins • Interview Analysis 1, n=20 • Survey analysis 2, n~400 • Survey Analysis 1, n~200

12. Audience Analysis • Goals • Inform our own design and evaluation • Contribute to broader discussion • Decisions • Dimensions? • Inform design, Speak to team, Theoretical traditions? • What methods? • Breadth/Depth, Acknowledge distributed nature of users

13. Dimensions Roles Goals Knowledge Circumstances of Use Culture Ergonomics Theoretical Perspectives Technical Communication Reader Response Theory Cognitive Science Constuctivism Distributed Intelligence Situated cognition Socio-Cultural Theory Human Factors Multidisciplinary Influences

14. Defining Dimensions • Role – Dominant persona of users (job, affiliation) • Goals – Reason for the interaction • Knowledge – The extent and nature of prior relevant knowledge • Circumstances of Use – Setting, resources, strategy, timing • Culture – Group level beliefs, language, preferences • Ergonomics – Relevant perceptual & motor abilities, skills

15. Method – Online Survey • Questions: Adaptive, ~25 questions • Participants: • Duration: 9/1/2000 – 7/2/2001 (10 months) • 472 respondents / 710 starts • Analyses1 – • Descriptive Statistics • Content Analysis • Qualitative Coding • Statistical Analysis • 1Acknowledgments: Tracey Wagner, Kristina Liu, Alice Tanada, Kristen Schuyler

16. About Visit Could you tell me about your visit or visits to the Arthritis Source? Could you tell me what you were trying to do when you visited the Arthritis Source? Did you benefit from your visit or visits to the Arthritis Source? What kind of information do you think other arthritis patients should know? About Knowledge of Condition Could you tell me what you think arthritis is in general? Could you tell me how RA/OA affects the body? Do you know what contributes to getting RA/OA? Do you know how RA/OA is diagnosed? If no, Do you remember what your doctor told you about your diagnosis? What is most difficult to understand about RA/OA? Method - Phone Interview

17. Phone Interview • Participants – 20 users (10 OA, 10 RA) • Analyses1 • Conceptions/misconceptions • Overarching Goals • Specific Information Needs • 1Acknowledgments: Tracey Wagner, Kristina Liu

18. Mapping Data to Categories

19. Results - Overview • What we learned… • 19% international (culture) • 26% rural (circumstances of use) • 21% over 60 (ergonomic, through vision implications) • Highlight specific examples where results had identifiable impact on design • Role • Goals • Knowledge • Circumstances of Use

20. Users with many roles “Person with arthritis” is too simplistic… Person with pain Person with condition that they do not consider arthritis Person who is exploring whether they have arthritis Roles n=462/472

21. Role – Design Implications • Added new types of information • Differential diagnosis • Identified writing guidelines… • Avoid statements that assume reader has the particular condition… (avoid -- “your condition” or “you need to…”)

22. Knowledge – Misconceptions? • From interview data, we identified several possible misconceptions: • Low bone density is associated with Osteoarthritis • Not drinking enough milk increases the risk of Osteoarthritis • Bone spurs cause arthritic pain • Osteoarthritis causes bone erosion. • OA is caused by the "wear and tear" of the joints due to overuse and aging. • There is little you can do

23. Knowledge – Design Implications • Added material to confront misconceptions • Added “common myths” question to generic templates • Developed Osteoarthritis knowledge quiz • 2513 Respondents for question 1 • 470 Respondents for entire 7-question quiz

24. Goals – Social Support?

25. Goals – Design Implications • On providing access to social support • Reduced priority for this within our system • Added content to help users find to support • On learning from a surprise • 20% of statements were direct questions • Indicates need to help users start with own questions • Question-based search permits this directly • Templates are based on user questions

26. Coming from… Implications Improve placement of site on search engines Orient users coming from search engines - accelerated implementation of the navigation based on the article templates. Circumstances of Use - Virtual n=372/472

27. Ongoing Audience Research • Ongoing analysis of user questions (goals) • From emails • From question-based search • Collected earlier… • Studying use over time through user online journaling (goals, circumstances of use, knowledge and learning)

28. Where we’ve been • Starting Point: When characterizing users/audience… • Which dimensions are most significant? • Which methods under which circumstances? • How will the insights inform design? • What are realistic expectations? • Contribution here: Case Study…

29. Moving beyond case studyFraming audience analysis decisions • Effort: Manager perspective • Resources to design, collect, analyze, interpret, decide • Insight: Researcher perspective • Rigorousness, representativeness, triangulation? • Impact: Designer perspective • Clarity of links to design, Persuasiveness of the data

30. Designing Audience Analysis • Effort: Manager perspective • Resources to design, collect, analyze, interpret, decide • Insight: Researcher perspective • Rigorousness, representativeness, triangulation? • Impact: Designer perspective • Clarity of links to design, Persuasiveness of the data Observation: Opportunities lie here…

31. Web-based Health Information • Site quality • Owner credentials, update dates (Hoffman, 2000) • Quality of information • Comprehensiveness (e.g., Chen, 2000) • Accuracy (e.g., Chen, 2000) • Providing references (e.g., Hellawell, 2000) • Findability of information • Time required (e.g., Gotwald, 2000) • Getting to real questions (e.g., Lechner, 1996) • Need for evaluation methods • (e.g., Wu, 2000, Delamsthe, 2000, Charatan, 1999)

32. When users are learners? • Characteristics of learners • Growth (Soloway, 1994) • Diversity (Soloway, 1994) • Motivation (Soloway, 1994) • Prior Understandings (NRC, 1999) • Thinking about implications • What to know and why • Knowing users over time • Variations (how many to know) • Time required to truly “know” Users Learners