Check Yes or No : Creating a Well-Designed Survey Research Process

1. 1 Check �Yes� or �No�: Creating a Well-Designed Survey Research Process October 20, 2006 9:00 AM - 10:30 AM�� -brief overview -nuts & bolts -hipaa -Q&A-brief overview -nuts & bolts -hipaa -Q&A

2. 2 Case Research Community

3. 3 Objectives Review of Human Subject Research Definitions and Policies Review of IRB role Types of survey research How survey research is done Survey research challenges Things to Consider & Helpful Hints Survey Research References Survey Research Databases

4. 4 Definition of Research A systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. 45 CFR 46.202

5. 5 Definition of a Human Subject A living individual about whom an investigator conducting research obtains: data through intervention or interaction with the individual,  -or- identifiable private information. 45 CFR 46.202

6. 6 The Belmont Report Basic Ethical Principles: Respect for Persons Individual autonomy Protection of individuals with reduced autonomy Beneficence Maximize benefits minimize harms Justice Equitable selection of subjects & distribution costs and benefits

7. 7 Review of IRB Role The IRB is an administrative body established to protect the rights and welfare of human research subjects recruited to participate in research activities conducted under the auspices of the institution with which it is affiliated. IRB Guidebook � OHRP � Department of Health & Human Services

8. 8 Types of Survey Research In Person This also includes focus groups and group discussions. Can take place anywhere. Telephone The most popular means of data collection. Mail Slow return and subject to errors Could be costly Internet/Web-based Growing in popularity Useful, but not as reliable as one would think Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

9. 9 When did survey and public opinion research start . . . and why? The first known survey done in the United States was the U.S. Census of 1790. While some organized survey research was done between then and the early 1900's, the real growth of the industry came after WW II. ~ �Surveys and You�, CASRO website Survey research is the most commonly recognized and applied social science research methodology for studying large populations. ~ University of Illinois @ Chicago Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

10. 10 How is survey and public opinion research done? Most survey research simply involves surveys among a representative sample of individuals. Very carefully designed questionnaires, usually administered in a structured manner, are used to guide the interview. They could be about attitudes, needs or preferences. The questions could be �closed-end� types (for example �yes� or �no�) or they could be �open-ended� (�what do you think of. . .�). ~ �Surveys and You�, CASRO website Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

11. 11 How is survey and public opinion research done? There are no right or wrong answers in survey research. [Researchers should o]nly [be] interested in the opinions of the people being questioned. If done carefully, sampling is more efficient than interviewing everybody. Even the U.S. Census Bureau interviews a small, but very carefully chosen sample at the end of its regular Census, and uses the results to adjust the findings of the whole Census! ~ �Surveys and You�, CASRO website Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

12. 12 How is survey and public opinion research done? Smaller samples are obviously much more economical for research purposes and, because they are small, can be controlled better for fewer errors. A properly chosen sample can be perfectly representative of the whole. For example, doctors take only a sample of blood for a blood test. ~ �Surveys and You�, CASRO website Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

13. 13 Survey Research Challenges Paper/Manual Data Collection Training Who will be administering interviews/surveys? Are they trained by RI? Do they know how to take �no� for an answer? CREC-Certified? (not mandatory for CIs, but it can�t hurt) Recruitment Where and how would RI obtain names/contact information? Permission to use names, if applicable? Scripts, Flyers, Emails, etc. Follow-up: How, where & how often? Be careful not to coerce

14. 14 Survey Research Challenges Data Collection Informed consent and/or a waiver How will data be collected & would collection method be understood by subjects? Privacy and Confidentiality concerns Will subjects be able to change their responses? If so, how? If not, do subjects know that? Data Storage/Disposal Data must be stored separately from consent in locked drawer/office. PCs/Laptops/Jump drives must be password protected Destroy identifiable data/master lists ASAP

15. 15 Survey Research Challenges Electronic Data Collection Authorization to access websites or chat rooms. Does the site/room consider itself a private place? Does it have research policies? Permission from webmaster, listserv manager or owner Recruitment All advertisement must comply w/IRB guidelines Authenticate subject respondents. i.e. Provide PINs for subjects Data Collection Data should be encrypted Note that encryption standards vary

16. 16 Survey Research Challenges Server Administration Recommend a professionally administered survey server Server should have firewalls, limited access, and frequent security audits Data Storage/Disposal Store personal identifying information, email and IP addresses separate from data Encrypt and backup data Store in safe/secure location Properly destroy data ASAP If safeguards cannot be assured, inform subjects that complete confidentiality CANNOT be guaranteed

17. 17 Survey Research Challenges Internet Research Issues (NCPP � �Not reliable�) No clear evidence that samples are accurate Only 1out of 3 American adults have internet access No clear evidence that methodology worked How would you know your study asked for the right information? Most electronic studies permit participating more than once (i.e. American Idol) Ballot stuffing

18. 18 Survey Research Challenges Internet Research Issues (NCPP � �Not reliable�) Does a web-based organization have a good track record? Professional research company with a good reputation? Does a web-based organization comply w/NCPP, CASRO, or AAPOR codes & standards of conduct? No different than Paper/manual standards Is a web-based organization willing to disclose questions & methodology? Can you explain it; withstand scrutiny

19. 19 Rights of Survey Participants Every Belmont Report principle applies Informed consent Subjects appropriately informed about survey�s intentions and how their personal information and survey responses will be used and protected; Participation is voluntary Protect subject identifiers Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

20. 20 Rights of Survey Participants Compensation � Is it pro-rated? Special care interacting w/vulnerable populations Is there a power differential? If so, how will that be addressed? Should you be RI? Verify researcher credentials Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

21. 21 Researcher Responsibilities Understand privacy & confidentiality How will you protect subjects & data? Be prepared to justify results To IRB, Integrity Boards, Publishers Research �trumps� marketing Regulatory agencies understand research standards are higher Be clear about what will be done w/subject info Informed consent is a process! Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

22. 22 Researcher Responsibilities How will final results be disseminated? Survey sponsor When & how was data collected? Sampling method Population sampled Consistent wording and questions; and understandable language Percentage of responses Was there follow up? Incentives for surveyors &/or participants Face to face include -Interviews -Focus groups/Group Discussion -ObservationFace to face include -Interviews -Focus groups/Group Discussion -Observation

23. 23 Things to Consider Privacy & Confidentiality Privacy - having control over the extent, timing, and circumstances of sharing oneself (physically, behaviorally, or intellectually) with others. Confidentiality - methods used to ensure that information obtained by researchers about their subjects is not improperly divulged.

24. 24 Things to Consider Using Secondary Data Secondary data are data that already exists prior to the initiation of research. Secondary data sets are considered �identifiable private information� if they contain individual identifiers. Data about employer, job title, age and gender may not individually identify a subject, but, when combined, could in certain cases identify a specific individual.

25. 25 Things to Consider Using Secondary Data If the information does not include individual identifiers and can not be linked to a living individual (anonymous) by the investigators conducting the secondary data analysis (RI or student), or is considered public or is given with the expectation that it will be made public and that it will be linked to the individual (i.e., biography, news story), then it would not be considered research involving identifiable private information and not require an IRB application as it is not human subject research. The IRB determines if an application is required.

26. 26 Things to Consider Standardized Questions = Usable Responses Questions have to be asked in exactly the same way of everybody--even though certain questions may be more interesting, or seem more relevant, to some people than to others. Changes in questions could cause a situation where a specific question may not seem meaningful, or is difficult to answer, to a specific person. ~ �Surveys and You�, CASRO website

27. 27 Things to Consider Know your research and methodology � Have a clear goal. *Carefully develop the questions. In what order will they be asked? Pretest questionnaires and procedures to identify problems prior to the survey � CANNOT KEEP DATA! Train interviewers carefully on interviewing techniques and the subject matter of the survey. Maximize cooperation or response rates within the limits of ethical treatment of human subjects � Know the potential ethical issues associated with your research Carefully develop and honor confidentiality given to respondents. ~ Standards & Best Practices, AAPOR 2002 * National Council on Public Polls

28. 28 Helpful Hints Plan ahead What is your goal? Do you plan to publish? Dissertation, Thesis or Classroom? Use correct & current version of the IRB forms Answer all application questions fully & completely

29. 29 Helpful Hints Attach all applicable documents � including online documents Contact the IRB if you have any questions Be prepared to receive feedback from the IRB

30. 30 Survey Research References American Association of Public Opinion Research www.aapor.org Council of American Survey Research Organizations www.casro.org/ National Council of Public Polls www.ncpp.org Great information on how to explain work to the press/public

31. 31 Survey Research References Survey Research Laboratory: University of Illinois at Chicago http://social-sciences.uchicago.edu/survey-lab/ ESOMAR � The World Association of Research Professionals www.esomar.org U.S. Census Bureau www.census.gov

32. 32 Survey Research Databases The Roper Center for Public Opinion Researchhttp://www.ropercenter.uconn.edu OhioLINK databasehttp://www.ohiolink.edu/resources.cgi Community Research and Development Information Servicehttp://www.cordis.lu/en/home. html

33. 33 Survey Research Databases Fedstatshttp://www.fedstats.gov The Washington Research Library Consortiumhttp://www.wrlc.org Case Linkhttp://msass.cwru.edu/library /ref/journal/resdbs.asp

34. 34 Contact Information Case Behavioral/Social Science IRB Administrative Office Office of Research Compliance, Sears Building, 6th Floor Location Code: 7230 cwru-irb@cwru.edu IRB Director: Isabel S�nchez ias5@cwru.edu 368-6993 IRB Assistant: Maureen Dore-Arshenovitz mxd4@po.cwru.edu 368-6925