designing a questionnaire

1. designing a questionnaire

2. Objectives in designing questionnaires • There are 3 main objectives are: • To maximise the proportion of subjects answering our questionnaire - that is, the response rate. • To obtain accurate relevant information for our survey. • In order to obtain accurate relevant information, we have to give some thought to what questions we ask, how we ask them, the order we ask them in, and the general layout of the questionnaire. • To maximise our response rate, • we have to consider carefully how we administer the questionnaire, establish rapport, explain the purpose of the survey, and remind those who have not responded. The length of the questionnaire should be appropriate.

3. Deciding what to ask There are three potential types of information: (1) Information we are primarily interested in-that is, dependent variables. (2) Information which might explain the dependent variables-that is, independent variables. (3) Other factors related to both dependent and independent factors which may distort the results and have to be adjusted for - that is, confounding variables.

4. Qualities of a Good Question • Evokes the truth. • Questions must be non-threatening. • When a respondent is concerned about the consequences of answering a question in a particular manner, there is a good possibility that the answer will not be truthful. • Anonymous questionnaires that contain no identifying information are more likely to produce honest responses than those identifying the respondent. • If your questionnaire does contain sensitive items, be sure to clearly state your policy on confidentiality.

5. Asks for an answer on only one dimension or only one piece of information at a time(NO DOUBLE BARREL QUESTIONS) The purpose of a survey is to find out information. A question that asks for a response on more than one dimension will not provide the information you are seeking. For example, a researcher investigating a new food snack asks: "Do you like the texture and flavor of the snack?" If a respondent answers "no", then the researcher will not know if the respondent dislikes the texture or the flavor, or both.

6. Another question asks, "Were you satisfied with the quality of our food and service?" Again, if the respondent answers "no", there is no way to know whether the quality of the food, service, or both were unsatisfactory. A good question asks for only one "bit" of information. Another example, "Please rate the lecture in terms of its content and presentation" asks for two pieces of information at the same time. It should be divided into two parts: "Please rate the lecture in terms of (a) its content, (b) its presentation."

7. Can accommodate all possible answers. Multiple choice items are the most popular type of survey questions because they are generally the easiest for a respondent to answer and the easiest to analyze. Asking a question that does not accommodate all possible responses can confuse and frustrate the respondent. For example, consider the question:

8. What brand of computer do you own? __A. IBM PCB. Apple Clearly, there are many problems with this question. What if the respondent doesn't own a microcomputer? What if he owns a different brand of computer? What if he owns both an IBM PC and an Apple? There are two ways to correct this kind of problem.

9. The first way is to make each response a separate dichotomous item on the questionnaire. For example: Do you own an IBM PC? (circle: Yes or No) Do you own an Apple computer? (circle: Yes or No)

10. Another way to correct the problem is to add the necessary response categories and allow multiple responses. This is the preferable method because it provides more information than the previous method. What brand of computer do you own?(Check all that apply) __ Do not own a computer__ IBM PC__ Apple__ Other

11. Has mutually exclusive options. A good question leaves no ambiguity in the mind of the respondent. There should be only one correct or appropriate choice for the respondent to make. An obvious example is: Where did you grow up? __ A. countryB. farmC. city A person who grew up on a farm in the country would not know whether to select choice A or B. This question would not provide meaningful information. Worse than that, it could frustrate the respondent and the questionnaire might find its way to the trash.

12. Produces variability of responses. When a question produces no variability in responses, we are left with considerable uncertainty about why we asked the question and what we learned from the information. If a question does not produce variability in responses, it will not be possible to perform any statistical analyses on the item. For example: What do you think about this report? __ A. It's the worst report I've readB. It's somewhere between the worst and bestC. It's the best report I've read

13. Since almost all responses would be choice B, very little information is learned [It's somewhere between the worst and best]. Design your questions so they are sensitive to differences between respondents. As another example: Are you against drug abuse? (circle: Yes or No) Again, there would be very little variability in responses and we'd be left wondering why we asked the question in the first place.

14. Follows comfortably from the previous question. Writing a questionnaire is similar to writing anything else. Transitions between questions should be smooth. Grouping questions that are similar will make the questionnaire easier to complete, and the respondent will feel more comfortable. Questionnaires that jump from one unrelated topic to another feel disjointed and are not likely to produce high response rates

15. Does not presuppose a certain state of affairs. Among the most subtle mistakes in questionnaire design are questions that make an unwarranted assumption. An example of this type of mistake is: Are you satisfied with your current auto insurance? (Yes or No) This question will present a problem for someone who does not currently have auto insurance. Write your questions so they apply to everyone. This often means simply adding an additional response category. Are you satisfied with your current auto insurance? ___ Yes___ No___ Don't have auto insurance

16. One of the most common mistaken assumptions is that the respondent knows the correct answer to the question. Industry surveys often contain very specific questions that the respondent may not know the answer to. For example: What percent of your budget do you spend on direct mail advertising? ____ Very few people would know the answer to this question without looking it up, and very few respondents will take the time and effort to look it up. If you ask a question similar to this, it is important to understand that the responses are rough estimates and there is a strong likelihood of error.

17. Does not imply a desired answer. The wording of a question is extremely important. We are striving for objectivity in our surveys and, therefore, must be careful not to lead the respondent into giving the answer we would like to receive. Leading questions are usually easily spotted because they use negative phraseology. As examples: Wouldn't you like to receive our free brochure? Don't you think the government is spending too much money?

18. Does not use emotionally loaded or vaguely defined words. This is one of the areas overlooked by both beginners and experienced researchers. Quantifying adjectives (e.g., most, least, majority) are frequently used in questions. It is important to understand that these adjectives mean different things to different people.

19. Does not use unfamiliar words or abbreviations. Remember who your audience is and write your questionnaire for them. Do not use uncommon words or compound sentences. Write short sentences. Abbreviations are okay if you are absolutely certain that every single respondent will understand their meanings. If there is any doubt at all, do not use the abbreviation. The following question might be okay if all the respondents are educated people , but it would not be a good question for the general public. What was your SES status? ______

20. Is not dependent on responses to previous questions. Branching in written questionnaires should be avoided. While branching can be used as an effective probing technique in telephone and face-to-face interviews, it should not be used in written questionnaires because it sometimes confuses respondents. An example of branching is: 1. Do you currently have a life insurance policy ? (Yes or No) If no, go to question 3 2. How much is your annual life insurance premium ? _________

21. Does not ask respondent to order or rank a series of more than five items. Questions asking respondents to rank items by importance should be avoided. This becomes increasingly difficult as the number of items increases, and the answers become less reliable. This becomes especially problematic when asking respondents to assign a percentage to a series of items. In order to successfully complete this task, the respondent must mentally continue to re-adjust his answers until they total one hundred percent. Limiting the number of items to five will make it easier for the respondent to answer.

22. The Order of the Questions • Items on a questionnaire should be grouped into logically coherent sections. • Grouping questions that are similar will make the questionnaire easier to complete, and the respondent will feel more comfortable. • Questions that use the same response formats, or those that cover a specific topic, should appear together

23. Each question should follow comfortably from the previous question. Writing a questionnaire is similar to writing anything else. Transitions between questions should be smooth. Questionnaires that jump from one unrelated topic to another feel disjointed and are not likely to produce high response rates

24. Arranging the questions The order of the questions is also important. Some general rules are: Go from general to particular. Go from easy to difficult. Go from factual to abstract. Start with closed format questions. Start with questions relevant to the main subject. Do not start with demographic and personal questions.

25. It is useful to use a variety of question format to maintain the respondents' interest. When a series of semantic differential scales are used, it may be a good idea to mix positive negative - for example, interesting to dull - with negative positive - for example, useless to useful - scales. This might make the respondents think more and avoid the tendency to tick the same response for every question.

26. Question Wording • The wording of a question is extremely important. Researchers strive for objectivity in surveys and, therefore, must be careful not to lead the respondent into giving a desired answer. Unfortunately, the effects of question wording are one of the least understood areas of questionnaire research.

27. Many investigators have confirmed that slight changes in the way questions are worded can have a significant impact on how people respond. Several authors have reported that minor changes in question wording can produce more than a 25 percent difference in people's opinions

28. Several investigators have looked at the effects of modifying adjectives and adverbs. Words like usually, often, sometimes, occasionally, seldom, and rarely are "commonly" used in questionnaires, although it is clear that they do not mean the same thing to all people. Some adjectives have high variability and others have low variability. The following adjectives have highly variable meanings and should be avoided in surveys: a clear mandate, most, numerous, a substantial majority, a minority of, a large proportion of, a significant numberof, many, a considerable number of, and several. Other adjectives produce less variability and generally have more shared meaning. These are: lots, almost all, virtually all, nearly all, a majority of, a consensus of, a small number of, not very many of, almost none, hardly any, a couple, and a few.

29. Use short and simple sentences Short, simple sentences are generally less confusing and ambiguous than long, complex ones. As a rule of thumb, most sentences should contain one or two clauses. Sentences with more than three clauses should be rephrased.

30. Avoid negatives if possible Negatives should be used only sparingly. For example, instead of asking students whether they agree with the statement, "Small group teaching should not be abolished," the statement should be rephrased as, "Small group teaching should continue." Double negatives should always be avoided.

31. Ask precise questions Questions may be ambiguous because a word or term may have a different meaning. For example, if we ask students to rate their interest in "medicine," this term might mean "general medicine" (as opposed to general surgery) to some, but inclusive of all clinical specialties (as opposed to professions outside medicine) to others.

32. Another source of ambiguity is a failure to specify a frame of reference. For example, in the question, "How often did you borrow books from your library?" the time reference is missing. It might be rephrased as, "How many books have you borrowed from the library within the past six months altogether?"

33. Ensure those you ask have the necessary knowledge For example, in a survey of university lecturers on recent changes in higher education, the question, "Do you agree with the recommendations in the Report on Higher Education" is unsatisfactory for several reasons. Not only does it ask for several pieces of information at the same time as there are several recommendations in the report, the question also assumes that all lecturers know about the relevant recommendations.

34. Level of details It is important to ask for the exact level of details required. On the one hand, you might not be able to fulfil the purposes of the survey if you omit to ask essential details. On the other hand, it is important to avoid unnecessary details. People are less inclined to complete long questionnaires. This is particularly important for confidential sensitive information, such as personal financial matters or marital relationship issues.

35. Handling Sensitive Issues It is often difficult to obtain truthful answers to sensitive questions. Clearly, the question, "Have you ever copied other students' answers in a degree exam?" is likely to produce either no response or negative responses. Less direct approaches have been suggested. Firstly, the casual approach: "By the way, do you happen to have copied other students' answers in a degree exam?" may be used as a last part of another decoy question. Secondly, the numbered card approach: "Please tick one or more of the following items which correspond to how you have answered degree examination questions in the past." In the list of items, include "copy from other students" as one of many items.

36. Thirdly, the everybody approach: "As we all know, most university students have copied other students' answers in degree exams. Do you happen to be one of them?" Fourthly, other people approach. This approach was used in the recent medical student survey. In this survey, students were given the scenario, “Jalil copies answers in a degree exam from Jamal." They were then asked, "Do you feel Jalil is wrong, what penalty should be imposed for Jalil, and have you done or would you consider doing the above?"

37. Length of questionnaire There are no universal agreements about the optimal length of questionnaires. It probably depends on the type of respondents. However, short simple questionnaires usually attract higher response rates than long complex ones. In a survey of stroke survivors both the response rate and the proportion of completed forms were higher for a shorter questionnaire (six questions with a visual analogue scale) compared with a longer and more complex questionnaire (with 34 questions).

38. Write in everyday terms.Avoid internal jargon. Many corporations have abbreviations or acronyms for products and services which are not familiar to custome Follow good business writing practices.Write short, simple questions. Be clear and to the point. Avoid errors in spelling, grammar and usage.

39. Use consistent scales. All rating scales should mimic the first one used. It can confuse respondents if you change from, for example, a five point to a seven point scale. Keep the scales going the same way. In other words, if '5' is high on the first scale, don't make '1' high on the next. Use similar wording for the anchors. Finally, group like questions under the same scale. If you do need to change scales, wait until you reach a new section of the questionnaire.

40. Use consistent wording. The use of similar phrases for the text of the survey can unify your questionnaire. For example, questions can be set up with a lead phrase which is a phrase that can be used to lead off each question. For example: How satisfied are you that our staff is: Responsive to your service requests ....... Knowledgeable about products ............. Knowledgeable about your business........

41. Avoid asking more than one question at a time. This is known as asking a 'double barreled' question. A typical double barreled question: "Sales reps are polite and responsive." While the sales reps may be polite they may not be responsive, or vice versa. The respondent will be forced to rate one attribute differently from their true feelings. Consequently, data interpretation will be questionable.

42. Provide directions. It is important to let the respondent know what to do on any particular question; however, it is just as important to avoid complicated directions. Make the survey as easy as possible for your respondents by using phrases such as 'Mark all that apply,' and 'Mark only one.' Avoid asking them to calculate anything, such as percentages, and try to avoid the use of skip patterns.

43. Analysis of the responses and the interviewers' comments are used to improve the questionnaire. Ideally, there should be sufficient variations in responses among respondents; each question should measure different qualities - that is, the responses between any two items should not be very strongly correlated - and the non-response rate should be low. In the third phase the pilot test is polished to improve the question order, filter questions, and layout.

44. Format of responses

45. Format of responses • The responses can be in open or closed formats. In an open ended question, the respondents can formulate their own answers. In closed format, respondents are forced to choose between several given options. What are the advantages of each of these formats? • It is possible to use a mixture of the two formats- for example, give a list of options, with the final option of "other" followed by a space for respondents to fill in other alternatives. • There are several forced choice formats. Out of these formats, ranking is probably least frequently used, as the responses are relatively difficult to record and analyse.

46. Closed-that is, forced choice-format • Easy and quick to fill in • Minimise discrimination against the less literate (in self administered questionnaire) or the less articulate (in interview questionnaire) • Easy to code, record, and analyse results quantitatively • Easy to report results

47. Example "How satisfied are you with your job?" (Circle the number that represents your response) Very disatisfied    Dissatisfied    Neutral    Satisfied    Very satisfied      1                  2                3          4             5

48. Example "What is your marital status?" (Check the box that applies) Single, never married Married Divorced Separated Widowed Other:_____

49. Please cicle your respons

50. Open format • Advantages • Allows exploration of the range of possible themes arising from an issue • Can be used even if a comprehensive range of alternative choices cannot be compiled