1 / 13

Survey Research

Survey Research. Considerations of Survey Research. Surveys produce close estimates of what people think or do (at best) More accurate surveys would involve Every member of population having equal chance of being selected Coverage error

metta
Download Presentation

Survey Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Survey Research

  2. Considerations of Survey Research • Surveys produce close estimates of what people think or do (at best) • More accurate surveys would involve • Every member of population having equal chance of being selected • Coverage error • Large enough sample to achieve level of precision (i.e., reduce sampling error) • Sampling error • Use of clear, unambiguous questions • Measurement effor • Everyone responds (or nonrespondents are similar to respondents) • Nonrespondent error

  3. Mail surveys (paper or web) Strengths: Require least amount of resources Minimize sampling error at low cost Provides more anonymity Weaknesses: Sensitivity to coverage error Can produce most nonresponses Researcher lacks control over process Face-to-face interviews Strengths: Avoids pitfalls of other methods (i.e., reading, no phone) Weaknesses: Higher cost Need good interviewers Telephone interviews Strengths: Produces results quickly Greater interviewer control Weaknesses: Exclusion of individuals, particularly certain populations Sample challenged by incomplete information Need knowledgeable interviewers Considerations of Survey Research

  4. Paper vs. Web Survey • Benefits/Advantages associated with web-based surveys • Cost-savings (printing, mailing, and data entry) • Instantaneous distribution • Skip-logic and other features • i.e. Large array of design options/features • Faster survey return rates • Reduced data entry time

  5. Paper vs. Web Survey • Challenges/Disadvantages associated with web-based surveys • Technology (problems with or limited access to) • Impact on sample or return rate • Perceived concern with anonymity • Mistakes in technology (i.e. code) can impact the results and responses

  6. Paper vs. Web Survey • Previous literature • Lower response rates found for electronic surveys (web or e-mail) • Includes failure to complete the survey • Mixed results over which mode lends to the most “skipped” or “missing” data • One needs to consider the technology skills of the user and the technology of the user’s computer • Web-surveys must consider coverage error • Presumed to have lower cost and take less time, but not always the case (must consider the entire survey process) • Responses may not be “the same” for the different modes • Some research suggested more positive responses with web-surveys

  7. Things to consider with web surveys • Screen-by-screen vs. scrolling • Research suggests improved response rates for multiple questions per screen • Completion time faster, fewer non-answered questions, more similarity in answers • Plain vs. fancy web-survey • Research indicated that higher responses rates were obtained for plain surveys • First question • Research has shown that this is a major drop-out point for many respondents • Open-versus closed-ended questions • Research suggests greater drop-out for open-ended questions

  8. Research suggests: Do: Keep survey brief and concise Place confidential or personal information at the end of the survey Have response categories in progressive order (lowest to highest) Write an introduction to the survey Use filtering questions Divide surveys into sections Use a convention similar to paper surveys Provide adequate instructions Pay close attention to physical layout Provide incentives Don’t (i.e. avoid): Use open-ended questions Have the response category of “other” Use response scale proliferation (i.e. 6 or 7 point scale) Ask respondents to rank responses Design long (excessive) surveys Overuse “fancy” features Things to consider with web surveys

  9. Survey Construction • Determine the problem you are trying to solve and the new information needed to solve it • Design questions that are NOT vague, biased, or lacking connection to solving the problem • Determine question type: • Open-ended vs. close-ended (and partially close-ended) • Category order-effect (with long-list of categories) • Consider the exact wording of questions • Consider field-testing your survey

  10. Survey Construction • Design attractive questionnaires • Consider the format and the printing • Consider your title (stimulate interest) • Consider your order of questions • One method: Most interesting to most objectionable • Start the 1st question with something NOT: open-ended, difficult, embarrassing, or about the individual

  11. Survey Construction • Determine your sample size… what sampling error are you willing to tolerate • Simple Designs: • Random sampling • Purposive sampling • Non-random sampling (i.e., a particular member of a household) • More complicated designs • Weighted sample (i.e., disproportionate sampling) • Two-state cluster sampling

  12. To improve response rate • Research suggests: • E-mail cover letters • Follow-up reminders via e-mail • Pre-notification with intent of survey • Simpler formats • Plain design • Mixed mode surveys (allow paper or web) • Shorter surveys

  13. References Bradley, R. V., & Sankar, C. S. (n.d.). Outcomes assessment: Electronic surveys versus paper-based surveys. ASEE Southeast Section Conference. Carini, R. M., Hayek, J. C., Kuh, G. D., Kennedy, J. M., & Ouimet, J. A. (2003). College student responses to web and paper surveys: Does mode matter? Research in Higher Education, 44, 1-19. Chatman, S. (2002). Going beyond the conversion of paper survey forms to web surveys. Student Affairs Online, 3. Retrieved November 21, 2005, from http://www.studentaffairs.com/ejournal/Winter_2002/surveys.html. Dillman, D. A., Tortora, R. D., Conradt, J., Bowker, D. (n.d). Influence of plain vs. fancy design on response rates for web surveys. Gunn, H. (2002). Web-based surveys: Changing the survey process. First Monday, 7(12). Retrieved November 21, 2005, from http://www.firstmonday.org/issue7_12/gunn/. Salant, P., & Dillman, D. A. (1994). How to conduct your own survey. John Wiley & Sons, Inc: New York. Smither, J. W., Walker, A. G., & Yap, M. K. T. (2004). An examination of the equivalence of web-based versus paper-and-pencil upward feedback ratings: rater-and ratee-level analyses. Educational and Psychological Measurement, 64, 40-61. Solomon, D. J. (2001). Conducting web-based surveys. Eric Digest. Retrieved November 21, 2005, from http://www.ericdigests.org/2002-2/surveys.htm

More Related