1 / 17

Constructing the questionnaire

Constructing the questionnaire. Survey Research and Design Spring 2006 Class #10 (Week 11). Today’s objectives. To answer questions you have To discuss analytic exercise #2 (and #3) To understand questionnaire design and ways to construct questionnaire

owen
Download Presentation

Constructing the questionnaire

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Constructing the questionnaire Survey Research and Design Spring 2006 Class #10 (Week 11)

  2. Today’s objectives • To answer questions you have • To discuss analytic exercise #2 (and #3) • To understand questionnaire design and ways to construct questionnaire • To spend time applying information to several surveys Survey Resarch and Design (Umbach)

  3. Primary objectives of design • Reduce nonresponse error and measurement error • Social exchange theory • Improve rewards by making it look interesting and important • Decrease costs by making it look easy to complete • Heuristics/satisficing • People tend to skim through surveys • Use simple rules of thumb to interpret what they see • End result: design is very important to a successful survey • Especially for mail and web surveys; they are self-administered Survey Resarch and Design (Umbach)

  4. Paper survey format • You should avoid • Double sided paper with a staple in the corner • Landscape orientation • Unusual/multiple folds (like the day care survey) • Unusual shapes (e.g., square) • Use a booklet form instead • People are used to booklet forms; less likely to get confused, skip pages • Legal size paper or 11” x 17”, folded and stapled on spine • Use two-column format for 11” x 17” • Shorter text lines are better • If you must, print single-sided and staple in corner • Dillman says this looks less professional than a booklet Survey Resarch and Design (Umbach)

  5. Web survey format • One page or multiple pages? • Depends if your software can handle multiple pages • Aldo how long your survey is – how long does it take to load when one page? • Avoid having too many separate pages; makes the survey seem long • Use some sort of progress indicator • Number of pages (1 of 4, 2 of 4) • Pie chart (20% complete, 40% complete) • Bar: • Make sure layout is vertical (reads from top to bottom) • Make sure the respondent does not have to scroll horizontally to read the survey questions Survey Resarch and Design (Umbach)

  6. Beginning and end of survey • Beginning • For paper, Dillman discusses possible design options • Include general instructions and survey sponsor • For web, these can appear at top of the first page • End • Dillman recommends a thank you and a space for comments • Include contact information for questions • Web surveys: include instructions for submitting as well as a submit button Survey Resarch and Design (Umbach)

  7. Question order • Avoid constantly switching topics • Try to sequence questions • By salience, with most salient at the beginning • Logically, e.g. by time • Group together questions that have similar components • Place sensitive and boring questions near the end of the survey • Less likely to object after filling out first part of survey • Examples: income, demographics Survey Resarch and Design (Umbach)

  8. Question order effects • Norm of evenhandedness or value-based effect • Anchoring or cognitive-based effect • Addition or carryover effect and subtraction effect • Increased positive rating of summary items when asked after specific items on the same subject • More likely to occur with opinion question and vague quantifiers Survey Resarch and Design (Umbach)

  9. Choosing the first question • Important because it gets the respondent “hooked” into survey • Four characteristics: • Easy to read and answer • Able to be answered by everyone • Interesting; should reference topic of survey described in contact • “Connectedness” Survey Resarch and Design (Umbach)

  10. Define a navigational path for reading all information on each page • Minimize the need to reread portions in order to comprehend response task • Place instructions where needed and not at beginning of survey • Place items with same response categories in groups • Saves space because response category is not repeated • But be aware, respondents will view items as a group • Avoid combining questions to save space • In higher ed, students are often asked to rate themselves on an item, and also rate how important that item is, at the same time • Minimize use of matrices (Dillman p. 103) Survey Resarch and Design (Umbach)

  11. Use visual elements as navigational guides • Increase font size to attract attention • Increase brightness or color to attract attention and establish groupings • Dillman bolds questions but not response categories • Use spacing and similarity to identify groupings • E.g, extra blank spaces between questions • Maintain consistent figure/ground format • Generally use black text on white or lightly shaded background • If shaded background, use white for answer spaces • Number questions and number them simply (1,2,3, etc.) Survey Resarch and Design (Umbach)

  12. Use visual elements as navigational guides • When possible, place special instructions inside question numbers, don’t have them stand alone (Dillman p. 119) • Maintain simplicity, regularity, and symmetry • See Dillman pp. 111-112 Survey Resarch and Design (Umbach)

  13. Visual design for questions • List answer categories vertically instead of horizontally; do not double- or triple-bank • Response categories should be evenly spaced • Important to check this for web surveys! • Response categories should be in correct order with conceptually neutral category in middle • Keep response scale direction consistent throughout survey • Separate at end of response category (with space or line) don’t know / not applicable responses • Think about size of text boxes for open-ended questions Survey Resarch and Design (Umbach)

  14. Visual design for questions • Use yes/no questions rather than check all that apply Please check all that apply: Yes No □ Library Library □ □ □ Athletic facilities Athletic facilities □ □ □ Dining hall Dining hall □ □ □ Computer lab Computer lab □ □ • Use alternating shaded rows or columns for large batteries of items, or periods to connect response category to check-box Survey Resarch and Design (Umbach)

  15. Skip instructions • These can be very problematic; require a lot of planning and testing • Directional arrows are key • Use a different type of directional arrow to indicate another path versus a skip • Increase font size of skip directions • Repeat qualifying answer in parentheses (e.g., “If yes”) • Indent the screened question Survey Resarch and Design (Umbach)

  16. Review surveys • Paper • Survey of Earned Doctorates • http://www.nsf.gov/statistics/srvydoctorates/survey2004/sed_2004.pdf • National Survey of College Graduates • http://www.nsf.gov/statistics/srvygrads/survey2003/grads_2003.pdf • Your First College Year • http://www.gseis.ucla.edu/heri/yfcy/06%20PDFs/YFCY2006.PDF • Web • College Student Experiences Questionnaire (CSEQ) – not complete • http://www.cseq.org/ • Faculty Survey of Student Engagement (FSSE) • http://nsse.iub.edu/pdf/fsse/FSSE_2006.pdf • Spend approximately 5 minutes to review each survey for design aspects (not question wording or order) Survey Resarch and Design (Umbach)

  17. For next class… • Readings: • Groves et al. – Chapters 9 & 11 • Dillman – Chapter 7 • AAPOR – Questions about IRBs • http://www.aapor.org/default.asp?page=survey_methods/IRBS_faq • NO CLASS NEXT WEEK. I know you’re disappointed.  Survey Resarch and Design (Umbach)

More Related