Questionnaire Design

1. Chapter 7 Questionnaire Design Learning Objectives: • Understand that questionnaire design is difficult, and understand why. • Explain the steps involved in designing an effective questionnaire. • Recognize how the method of data collection influences questionnaire design. • Understand the types of questions and how they are used. • Understand the three major sections of the questionnaire and how they relate to each other.

2. Questionnaire . . . . a prepared set of questions (or measures) to which respondents or interviewers record answers

3. Steps in Questionnaire Design: Step 1: Initial Considerations Step 2: Clarification of Concepts Step 3: Typology of a Questionnaire Step 4: Pre-testing of a Questionnaire Step 5: Administering a Questionnaire

4. Questionnaire Design – Initial Considerations: • Clarify the nature of the research problem and objectives. • Develop research questions to meet research objectives. • Define target population and sampling frame. • Determine sampling approach, sample size, and expected response rate. • Make a preliminary decision about the method of data collection.

5. Research Questions: • Is sexual harassment a problem in this organization? • Do employees in this organization support diversity in the workplace? • Does religious affiliation influence support for human cloning? • What are the most important factors influencing the purchase of a laptop computer? • What are the good and bad issues of President Bush’s policies on eliminating terrorism?

6. Self-Completion or Interviewer Assisted Questionnaire? Respondent capabilities: • Educational background. • Vocabulary level. • Prior experience in completing questionnaires. • Age. • Cultural issues.

7. Questionnaire Design – Clarification of Concepts: • Ensure the concepts(s) can be clearly defined. • Select the variables/indicators to represent the concepts. • Determine the level of measurement.

8. Questionnaire Design – Typology of a Questionnaire: • Determine the types of questions to include and their order. • Check the wording and coding of questions. • Decide on the grouping of the questions and the overall length of the questionnaire. • Determine the structure and layout of the questionnaire.

9. Open-ended Questions = place no constraints on respondents who are free to answer in their own words.Closed-ended Questions = respondent is given the option of choosing from a number of predetermined answers. QUESTIONNAIRE DESIGN Two Types of Questions: • Open-ended. • Closed-ended.

10. Open-Ended Questions: • What do you think about your health insurance plan? • Which mutual funds have you been investing in for the past year? • How are the funds you are investing in performing? • What do you think of airport security?

11. Open-ended Questions • Typically used in exploratory/qualitative studies. • Typically used in personal interview surveys involving small samples. • Allows respondent freedom of response. • Respondent must be articulate and willing to spend time giving a full answer. • Data is in narrative form which can be time consuming and difficult to code and analyze. • Possible researcher bias in interpretation. • Narrative can be analyzed using content analysis. Software is available (e.g., NUD*IST).

12. Closed-end Questions: • Single Answer. • Multiple Answer. • Rank Order. • Numeric. • Likert-Type. • Semantic Differential.

13. Close-Ended Questions: • Did you check your email this morning? __ Yes __ No • Do you believe Enron senior executives should be put in jail? __ Yes __ No • Should the United Kingdom adopt the Euro or keep the pound? • Adopt the Euro __ • Keep the pound __

14. Closed-end Questions: • Typically used in quantitative studies. • Assumption is researcher has knowledge to pre-specify response categories. • Data can be pre-coded and therefore in a form amenable for use with statistical packages (e.g., SPSS, SAS) – data capture therefore easier. • More difficult to design but simplifies analysis. • Used in studies involving large samples. • Limited range of response options.

15. Broad Considerations: • Sequencing of questions. • Identification of concepts. • How many questions are required to capture each concept? • Question wording. • Overall length of questionnaire. • Placing of sensitive questions. • Ability of respondents. • Level of measurement. • Open-ended versus closed-end questions.

16. Questionnaire Sections Go On-Line www.raosoft.com Opening Questions Research Topic Questions Classification Questions

17. Branching Questions Are used to direct respondents to answer the right questions as well as questions in the proper sequence.

18. Screening or Filter Questions: . . . are used to ensure respondents included in the study are those that meet the pre-determined criteria of the target population. “Tonight we are talking with individuals who are 18 years of age or older and have 50 percent or more of the responsibility for banking decisions in your household. Are you that person?” __ Yes __ No

19. Rapport Questions: . . . are used to establish rapport with the respondent by gaining their attention and stimulating their interest in the topic. • “Have you seen any good movies in the last month?” __ Yes __ No • “What is your favorite seafood restaurant?”

20. Concept = a generic idea formed in the mind. Example Concept: “Customer Interaction” This customer was easy to talk to. This customer genuinely appreciated my helping him/her. This customer was friendly. This customer seemed interested in me, not only as a salesperson, but also as a person.

21. Concepts Concept Identification. • Conceptual definition – e.g., Service Quality. As perceived by customers, it is the extent of discrepancy between customers’ expectations or desires and their perceptions. Working Definition for Concept. • Decompose definition into components. • Search for items that are measurable.

22. Preparing and Presenting Good Questions: • Use simple words. • Be brief. • Avoid ambiguity. • Avoid leading questions. • Avoid double-barreled questions. • Be careful about question order and context effects. • Check questionnaire layout. • Prepare clear instructions.

23. Avoid Position Bias: Position Bias: • “How important are flexible hours in evaluating job alternatives?” • “What factors are important in evaluating job alternatives?” No Position Bias: • “What factors are important in evaluating job alternatives?” • “How important are flexible hours in evaluating job alternatives?”

24. Double-Barreled Questions: To what extent do you agree or disagree with the following statements? • “Harrod’s employees are friendly and helpful.” • “Harrod’s employees are courteous and knowledgeable.”

25. Branching Questions: . . . are used to direct respondents to answer the right questions as well as questions in the proper sequence. • “Have you seen or heard any advertisements for wireless telephone service in the past 30 days?” • “If ‘No’, go to question #10. • “If ‘Yes’ , were the advertisements on radio or TV or both?” • “If the advertisements were on TV or on both radio and TV, then go to question #6? • “If the advertisements were on radio, then go to question #8.” Following questions #6 and #8 the next question would be: • “Were any of the advertisements for ‘Sprint PCS’?”

26. Self-Completion Instructions: The following issues typically are considered: • Introducing and explaining how to answer a series of questions on a particular topic. • Transition statements from one section (topic) of the questionnaire to another. • Which question to go to next (branching or skipping). • How many answers are acceptable, e.g., “Check only one response.” Or “Check as many as apply.” • Whether respondents are supposed to answer the question by themselves, or can consult another person or reference materials. • What to do when the questionnaire is completed, e.g., “When finished, place this in the postage paid envelope and mail it.”

27. Interviewer-Assisted Instructions: The following issues typically are considered: • How to increase respondent participation? • How to screen out respondents that are not wanted and still keep them happy? • What to say when respondents ask how to answer a particular question? • When concepts may not be easily understood, how to define them? • When answer alternatives are to be read to respondents (aided response) or not to be read (unaided response)? • How to follow branching or skip patterns? • When and how to probe? • How to end the interview?

28. Questionnaire Design – Pre-testing of a Questionnaire: • Determine the nature of the pretest for the preliminary questionnaire. • Analyze initial data to identify limitations of the preliminary questionnaire. • Refine the questionnaire as needed. • Revisit some or all of the above steps, if necessary.

29. Questionnaire Design – Administering a Questionnaire: • Identify the best practice for administering the type of questionnaire utilized. • Train and audit field workers, if required. • Ensure a process is in place to handle completed questionnaires. • Determine the deadline and follow-up methods.

30. So you think your respondents understand your questions ? Recently a survey was conducted by the United Nations using a sample from several different countries. One of the questions asked was: “Would you please give your opinion about the food shortage in the rest of the world?” The survey was a huge failure. Why? • In Africa they did not know what food meant. • In Western Europe they did not know what shortage meant. • In Eastern Europe they did not know what opinion meant. • In South America they did not know what please meant. • And in the United States they did not know what the rest of the world meant.

31. Questionnaire Design On-Line Decision Analyst www.decisionanalyst.com Perseus Development www.perseusdevelopment.com SPSS www.spss.com Socratic Technologies www.sotech.com

32. Questionnaire Design On-Line Survey Builder www.surveybuilder.com SurveyPro.com www.surveypro.com The Survey System www.surveysystem.com Websurveyor www.websurveyor.com

33. Questionnaire Design Go On-Line www.surveypro.com How does this website help you to design questionnaires?

34. Questionnaire Design Go On-Line www.customersat.com What did you learn about questionnaire design from this website?