Constructing Survey Questions

1. Constructing Survey Questions Notes for Social Sciences

2. Types of Survey Questions • Open-ended • Open-ended questions have no one definite answer. Ex.) Why is this issue important? • Pro: Yields quotable material • Con: Responses are more difficult to catalogue and interpret. • Closed-ended • Closed-ended questions ask participants to choose from a specific set of answers. • Pro: Easy to quantify • Con: Limited results, may not know the answer a respondent wants to give.

3. Styles of Closed-ended Questions Likert-scale: To assess a person's feelings about something. • How important do you think standardized test scores are to a fifth-grader's education: • Not very important Extremely important 1 2 3 4 5 Multiple-choice: • When there are a finite number of options, always give the option of “other.” Ordinal: To rate things • Example: Please write a number between 1 and 5 next to each item below. Put a 1 next to the item that is MOST important to you in selecting an on-line university course. Put a 5 next to the item that is LEAST important. Please use each number only ONCE. ___ a. Availability of instructor for assistance. ___ b. Tuition cost for the course. ___ c. Ability to work in groups with other students. ___ d. Quality and quantity of instructor feedback. ___ e. Number of students enrolled.

4. Sample Survey-Open-ended ?s • Research Question: Do students take recycling seriously? 1. How often do you recycle? ● All the time ● When it’s convenient ● Not at all 2. Do you feel that recycling is important to the environment? ● Very important ● Important ● Somewhat important ● Not important ● No opinion 3. What would you do if you saw someone not recycling? ● Say something to them● Take out their trash and recycle it for them ● Not say anything at all 4. Are people aware of the importance of recycling? ● Very aware ● Somewhat aware ● Not aware at all 5. What factors determine your likelihood to recycle? Convenience Cost Pressure from friends/family All of the above None of the above

5. Sample Survey-Closed-ended ?s • 1. What religion are you affiliated with? • 2. Are you in a relationship? And if so, what religion is that person affiliated with? • 3. Have you ever dated someone outside your religion? If not, would you? • 4. What barriers do you place on a relationship? • 5. Does your family or place of worship influence your decision on whom to date? • You will also need to decide if factors such as: age, sex, or race are relevant to your study. Older students may be more likely than younger students to date outside their religion. Maybe your study is examining how men and women perceive commitment differently. You need to think about these issues before you distribute your survey.

6. Types of questions to avoid • Biased questions • Biased questions are questions that encourage your participants to respond to the question in a certain way. They may contain biased terminology or are worded in a biased way. • Biased question: Don't you agree that campus parking is a problem? • Revised question: Is parking on campus a problem?

7. Types of questions to avoid, cont.. • Questions that assume what they ask • These questions are a type of biased question and lead your participants to agree or respond in a certain way. • Biased question: There are many people who believe that campus parking is a problem. Are you one of them? • Revised question: Do you agree or disagree that campus parking is a problem?

8. Types of questions to avoid, cont.. • Double-barreled questions • A double-barreled question is a one that has more than one question embedded within it. Participants may answer one but not both, or may disagree with part or all of the question. • Double-barreled question: Do you agree that campus parking is a problem and that the administration should be working diligently on a solution? • Revised question: Is campus parking a problem? (If the participant responds yes): Should the administration be responsible for solving this problem?

9. Types of questions to avoid, cont… • Confusing or Wordy Questions • Make sure your questions are not confusing or wordy. Confusing questions will only lead to confused participants, which leads to unreliable answers. • Confusing questions: • What do you think about parking? (This is confusing because the question isn't clear about what it is asking--parking in general? The person's ability to park the car? Parking on campus?) • Do you believe that the parking situation on campus is problematic or difficult because of the lack of spaces and the walking distances or do you believe that the parking situation on campus is ok? (This question is both very wordy and leads the participant.) • Revised question: What is your opinion of the parking situation on campus?

10. Types of questions to avoid, cont… • Questions that do not relate to what you want to learn • Be sure that your questions directly relate to what it is you are studying. A good way to do this is to ask someone else to read your questions or even test your survey out on a few people and see if the responses fit what you are looking for. • Unrelated questions: Have you ever encountered problems in the parking garage on campus? Do you like or dislike the bus system?

11. Survey Terminology • Biased data—Information that leads the respondent to answer in a particular way • Respondent—Someone who answers survey questions • Investigator—The person who administers the survey • Relevant subjects—People who have knowledge about the survey’s topics • Capability of respondents—The ability to accurately respond to the questions • Pilot testing—To measure the tendency of the survey to gather the information desired so that it can be rewritten as needed • Target population—The target population is the entire group a researcher is interested in, the group about which the researcher wishes to draw conclusions • Example: Suppose we take a group of men aged 35–40 who have suffered an initial heart attack. The purpose of this study could be to compare the effectiveness of two drug regimes for delaying or preventing further attacks. The target population here would be all men meeting the same general conditions as those actually included in the study. • Sample—A set of individual units, drawn from some definable population of units and generally a small proportion of the population, to be used for a statistical examination; the findings are intended to be applied to the population • Population—The largest group under study that includes all individuals meeting the defined characteristics • Validity—Survey measures what it says it is measuring

12. Sample Survey • Purpose: • This survey was designed to get an idea of how young, Indian women feel about their culture and its attitude in regards to their gender. Hindu beliefs and practices have often been regarded as traditional, with strict attitudes towards women. • Subjects • This survey was distributed to female college students in the United States of Indian descent. There was a total of 16 replies from women ranging in age from 18-26.

13. Analyze these questions • Age? • Were you born in the U.S.? • Siblings? Ages and Genders? • Have you ever dated someone who was not an Indian? • Do you plan to get married? Do you plan to marry an Indian? • Would you ever accept an arranged marriage? • Do you consider yourself traditional? • Have you tried to be more active in the community? • Have you experienced sexual discrimination through this culture?

14. Discuss • The End