1 / 17

The CSES Questionnaire Design Process

The CSES Questionnaire Design Process. David Howell University of Michigan dahowell@umich.edu Jessica Fortin GESIS – Leibniz Institute for the Social Sciences jessica.fortin@gesis.org European Survey Research Association (ESRA) Lausanne, Switzerland July, 2011. CSES Questionnaire.

norton
Download Presentation

The CSES Questionnaire Design Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The CSES Questionnaire Design Process • David Howell • University of Michigan • dahowell@umich.edu • Jessica Fortin • GESIS – Leibniz Institute for the Social Sciences • jessica.fortin@gesis.org • European Survey Research Association (ESRA) • Lausanne, Switzerland • July, 2011

  2. CSES Questionnaire • Over 60 countries participate • Common module of survey questions • new theme every five years • some questions the same • some different according to new theme • to be run “as is” • Administrative, demographic and voting variables to national standards

  3. Questionnaire Design Process

  4. Questionnaire Design Process • A lengthy development process • Over a three year period, 2008-2011 • Iterative • As transparent as possible • Collaborative, with broad input

  5. Planning Committee Formation A Planning Committee, comprised of, selected by, and informed by collaborators, designs and oversees each Module. • Public call for nominations • Nominating Committee recommends members • Plenary Session confirms list and suggests additional members • Result = 20 members from 15 countries

  6. Theme Development Every five years a new module is developed to specifically address a big question in science. • Multiple public calls for theme proposals • Task Force collects proposals • Task Force presents and evaluates proposals for Plenary Session and Planning Committee • Theme(s) tentatively decided upon before questionnaire design begins

  7. Questionnaire Development After a theme is arrived at, attention turns to developing the questionnaire for it. • Subcommittees formed to pursue the new theme(s) and improvements to past questions • Subcommittees return to proposers for revisions to their original proposals • Draft questionnaire produced • Planning Committee discussion, revision • Repeat as necessary

  8. Pretesting The draft questionnaire is pretested cross-nationally in different contexts. • Partial pretestsin Brazil, Germany, UK • Full pretestsin Canada, Ireland • Reports received from pretest countries

  9. Questionnaire Finalization After having received extensive feedback, the questionnaire can now be finalized. • Subcommittees formed to address sections outside of the new theme(s) • A Plenary Session provides final feedback • Review of pretest results, subcommittees • Planning Committee makes final changes • Final questionnaire disseminated

  10. Challenges

  11. Design Challenges • 10-15 minutes is not much questionnaire time … but, pressure to reduce collaborator burden • competing with other content on surveys • competing with other comparative studies • Existing, validated questions preferred • Priority to cross-national questions • must be applicable in most all countries • must work well across most all countries • must be feasible in a variety of study designs

  12. Information Challenges • A lot of people are involved • Managing and tracking lots of input • Reconciling many opinions and approaches • Consensus not always possible • Making decisions with imperfect information • More testing in more contexts would always be better, but relies on volunteers who... • Have an election coming up (optimally) • Have a funded survey with available space • Can produce reports in a timely fashion • Eventually decisions must be finalized

  13. Collective Challenges • A lot of countries are involved • Cross-national and cross-cultural applicability and equivalence • Accommodating the variety of institutional arrangements in the many countries • Difficult to predict or account for all situations, even with broad cross-national participation in the process

  14. Where to go from here

  15. Where to go from here • A formal post-process evaluation • We’ve done this a number of times now • What worked well, what didn’t • Consider improvements • Formalize and document process for next time Some possibilities: • More extensive pretesting • Allow more time for pretesting • Test in more cultures and contexts • Possible use of focus groups

  16. Where to go from here • More focus on cross-national/cultural issues ... concept equivalence ... collaborator/interview instructions ... translation issues • More methodological working groups • Learning from the approaches of other cross-national studies

  17. Thank you for your time! • To learn more about CSES, or to download data: • www.cses.org • ...or email your questions to: • cses@umich.edu

More Related