1 / 18

Mixed mode in LFS: questionnaire design and mode-effects

Mixed mode in LFS: questionnaire design and mode-effects. Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of May 2014. StatFin/EssNet web-project:. Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22)

aphrodite-stephenson
Download Presentation

Mixed mode in LFS: questionnaire design and mode-effects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15th of May 2014

  2. StatFin/EssNet web-project: • Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22) • Pilot survey (N = 8 000), experimental design (”split-plot”) with DK-option, working time and job search questions. Comparison to CAWI data from the same period. Paradata, respondent feedback.

  3. Content of the presentation: • Questionnaire design: results & recommendations • Data collectionprocess • Mode-effects: WSTATOR (employment status) • Mode-effects: HWACTUAL (actualworkinghours) • Qualitative feedback

  4. 1. Questionnaire design: layout • BlaiseIS default layout was not perceived as attractive • Number of question / page • Navigation bars • Instructions • Error messages • Don’t know –option (DK)

  5. Status bar Number of questions, updating the remaining questions Error message, DK Instructions, info-button Scrolling down for navigation bars

  6. When the cursor points the info-symbol, the instructions appear in a box below.

  7. Kyllä = Yes En = No (Don’t know not visible) Navigation bars: Takaisin = Back Jatka = Continue Jatka myöhemmin = Continue later/pause

  8. DK-option, version 1 An error message: Please respond to this question or choose the “Don’t know” option. Kyllä = yes Ei = no En osaa sanoa = Don’t know

  9. DK-option, version 2 Kyllä = yes Ei = no En osaa sanoa = Don’t know

  10. 2. Data collection process • Response rate 30 %  potential for savings in data collection costs when the sample is large enough • CAWI-data more ”distorted” ? CAPI non-response more evenly distriputed with regard to educational background, gender and age than CAWI-data. • Speed: 40 % responses were received withing 3 days  after that switch to CAPI?

  11. 3. Mode-effects: WSTATOR (employment status)

  12. 3. Mode-effects: WSTATOR (employment status) • No significant mode-effects, but…. • media attention remains if unemployment rate 8,2 % instead of 7,4 % • sample for pilot study was small • CAWI respondents answer more often that they have been looking for job • = mode effect or selection bias?

  13. 4. Mode-effects: HWUSUAL (actual working hours) ”One question” –format for HWACTUAL: “How many hours did you work in your main job during the week 7.10.-13.10.2013?” ___ hours ___ minutes

  14. ”Grid” –format for HWACTUAL: “How many hours did you work in your main job during the week 7.10.-13.10.2013?” Date Hours Minutes Monday 7.10.2013 ___ ___ Tuesday 8.10.2013 ___ ___ Wednesday 9.10.2013 ___ ___ Thursday 10.10.2013 ___ ___ Friday 11.10.2013 ___ ___ Saturday 12.10.2103 ___ ___ Sunday 13.10.29013 ___ ___ Total ___ ___

  15. Average number of hours worked (CATI-data and two different CAWI-datas)

  16. 5. Qualitative feedback • Too intimate information on employer • Questions do not fit for people with multiple jobs without main job or with scholarship • All kind of entrepreneurial activity is not covered • Working time items do not fit for people in atypical employment • Pensioners do not understand point of reference week

  17. Conclusions and lessons learnt • Response rate of 30 % and speed of data collection encourage to continue towards mixed-mode LFS. • Visual design is important: balanced layout, intuitive navigation bars, informative error messages, a few questions per page, hidden instructions, hidden DK. • CAWI data more ”distorted” • In working time questions different format for CATI and CAWI may be needed to reduce mode-effects (”functional equivalency”). Differences in WSTATOR need to be further analysed.

  18. Thank you! Contact: liisa.larja@stat.fi

More Related