Redirected Inbound Call Sampling (RICS) A New Methodology

1. Redirected Inbound Call Sampling (RICS)A New Methodology Presenter: Karol Krotki, RTI Co-Authors: Georgiy Bobashev, RTI Burton Levine, RTI Scott Richards, Reconnect Research Conference on Inference from Non-Probability Samples 16-17 March 2017, Paris, France

2. Survey Research Start – Face-to-Face

3. Next development – telephone surveys

4. BUT…now most people ONLY use cell

5. Can’t robo-call cell phones – expensive

6. Caller ID results in very low response rates

7. Much research moved online…

8. The phone died or became very expensive

9. MIDI calls Traditional telephone surveys use outbound calls. Redirected Inbound Call Sampling (RICS) uses inbound calls that fail to connect to the intended phone. M Misdialed I Incomplete D Disconnected number I Inbound

10. Volume of MIDI calls

11. There are billions of MIDI calls In the US there are over 150 billion calls a month. 5 billion are MIDI calls. Reconnect Research (RR) connects MIDI calls to surveys. • MIDI calls come from • Misdials to a bank of toll free numbers owned by RR • Routed from telecom companies • AT&T • Verizon

12. Funnel process

13. Survey conducted via IVR

14. Invitation

15. Screening for adults

16. RICS

17. RICS Advantages Advantages • Low cost (20 question study costs $4 per R, no screening) • Speed: 1,000 completed surveys in two hours • Large sample size • Can target billing ZIP Code • Single stage design • Consistency across time • Lower level of respondent burden • Respondent initiates the process

18. RICS Challenges Disadvantages • Nonprobability sample • Limitations of IVR • Survey has to be short (max. 15 mins.) • Programming issues • IRB issues

19. Validating RICS—first steps How does one validate a non-probability sample? Does the selection mechanism result in Coverage error/bias? Selection error/bias? Field an RICS and investigate the discrepancy between the demographic distribution of the respondents and census estimates, the weighted outcomes and a national probability study. Nonresponse error complicates our ability to quantify selection error.

20. Data Collection Results Data collection results by day Distribution of completion time—in minutes

21. Data Collection Results—continued 52,849 non-duplicated phone numbers were solicited 6,799 eligible respondents. *Eligibility requirement—respondent is 18+

22. Demographic Results

23. Demographic Results—continued

24. Calibration Unequal weighting effect = 1.14

25. Health Outcome Results

26. Compare Health Outcomes Weighted—Hours of Sleep

27. Health and Media SurveyQuick Assessment of Twitter use and Health • Do you have a Twitter account? • How often do you tweet? • In the past year, have you had flu or flu-like symptoms? • In the year, has anyone else in your household had flu or flu-like symptoms within 2-week period of yours • In the past year, did you get a flu vaccine? • Would you say that in general your health is excellent, very good, good, fair, or poor? • In the past year have you been admitted to an emergency room >2 times? • In the past week, have you smoked 4 or more cigarettes? • How many of your close friends and family are regular tobacco smokers? • How often do you smoke e-cigarettes, vape pens, or hookah pens?

28. Sequential Comparison • With another RICS survey • With a national outbound telephone survey • With a national household survey • With other sources of data

29. Results and MessagesReplicability with Another RICS Survey • Response rates: • RICS_BRFSS 14.5% • HMS 6.8%

30. Comparison with a Phone Survey BRFSS/census

31. Comparison with a National Household Survey NSDUH 2014. Weighted Analysis

32. Twitter Use: Pew and HMS

33. Rapid Biosurveillance (Flu last year, household transmission, vaccination, networks) • Flu past year (self): 13.9% • Flu Household within 2 week interval: 43.6% • Flu vaccinated last year 44.0% • NotSick (vaccinated vs. not) 83.2 vs. 84.3 • Sick (vaccintated vs. not) 13.5/13.9 • Close friend/family smoking • Smokers vs non-smokers: 55% vs. 37% Flu attack rate from Nature 2015 about 7-30%

34. Table 1: Sample NHIS and RICS Table 2: Duration of Interview in minutes

35. Table3: Flow of subjects through the study Data Process Flow

36. RICS - Sex

37. RICS - Age

38. RICS – Race/Ethnicity

39. RICS - Education

40. RICS – Census Divisioin

41. RICS vs. NHIS – Substantive Results

42. Projects Where RICS May Be Suitable RICS maybe particularly suited for projects with: • Very low budget • Short time lines • To sample special or hard to access populations (e.g. specific zip code, people with asthma) • Situations where outcomes change quickly • Surveillance of influenza • Public opinion • Situations where data are available to measure potential bias

43. Future Research • Adjust for selection bias, based on: • On average number of calls placed per day • Phone type • Apply a nonresponse adjustment based on ZIP code demographic characteristics. • Incentives: methodology and potential impact • Add, validate and compare MIDI calls from other data sources

44. Future Research—continued • Mode effect—IVR vs. live interview • Test redirection to other modes: e.g., live phone interview, online • Use RICS to sample rare populations • e.g. boaters, Floridians 18-24 year-old • Use RICS to recruit subjects to • A live telephone interviewer • A web survey

45. Conclusions • Redirected Inbound Call Sampling (RICS) is a new and promising tool for surveillance and more broad data collection • More statistical analysis is needed to provide rigor and inference from RICS • RTI International and Research Reconnect are partnering to provide scientific basis to RICS

46. Comments Questions Contact Information Karol Krotki RTI kkrotki@rti.org