1 / 120

Review of Survey Methodology

Review of Survey Methodology. Bradley A. Woodruff, MD MPH International Emergency and Refugee Health Branch, U.S. Centers for Disease Control and Prevention. Review of Survey Methodology. Review some basic principles in surveys and sampling Describe some issues which

Download Presentation

Review of Survey Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Review of Survey Methodology Bradley A. Woodruff, MD MPH International Emergency and Refugee Health Branch, U.S. Centers for Disease Control and Prevention

  2. Review of Survey Methodology • Review some basic principles in surveys and sampling • Describe some issues which • Often done incorrectly or inefficiently • Remain without strong consensus • Opinions expressed are only my own • Given to provide basis of discussion • Recommendations are strictly personal and not meant to be adopted by this meeting

  3. Review of Survey Methodology • When to use surveys • Sampling methods • Final sampling stage (in multistage sampling) • Sample size • Number of clusters • Select 1 or all eligible persons? • Judging age eligibility • Children <6 months of age • Measuring mortality • High mortality and low malnutrition

  4. When to use surveys

  5. When to use surveys A cross-sectional survey is a collection of data from a specific population at a single point in time.

  6. When to use surveys

  7. When to use surveys Alternate data collection methods • Morbidity and mortality surveillance • Birth and death registration • Qualitative methods • Program data • Others

  8. When to use surveys What a single survey can tell you • The point prevalence of health or nutrition outcome • Retrospective measure of cumulative incidence, such as mortality rates • Can assess outcomes when other data collection systems absent or impossible

  9. When to use surveys What a single survey cannot tell you • Cause-effect relationships • Why events occur or why things are the way they are • Trends over time • However, repeated surveys can measure change

  10. When to use surveys Purposes of a survey • Determine need for new program • Design new program • Evaluate existing program

  11. Strengths of surveys Can give reasonably accurate estimate of prevalence of health condition in population Can be replicated to evaluate health outcomes Can be done when other data collection systems (e.g. surveillance) not feasible Weaknesses of surveys Difficult to assess cause/effect Often confined to short, simple questions Difficult to answer “why” questions Must be repeated to follow trends over time Sometimes difficult to define population of interest Sampling can be complex and is often not done well   When to use surveys

  12. When to use surveys • Surveys cost money, manpower, and time • Collect only data useful for program purposes • If primary objective is measuring malnutrition prevalence or mortality rate • Do not add extraneous data collection • However, large part of resources spent in traveling to selected households

  13. When to use surveys – and where? • National-level data • Requires only one survey for country • Provides overall picture • Can serve as general assessment of nutritional status • NGOs often work on local level • Local data needed for program decisions • Difficult to formulate overall picture from many local surveys • Lack of standardization inhibits comparison • Difficult to achieve good coverage

  14. When to use surveys - recommendations • Before beginning preparations • Carefully consider if survey necessary before beginning • Identify specific uses of survey results • Design survey to answer specific questions • Rigidly follow correct sampling procedures

  15. Sampling methods

  16. Sampling methods • The goal of sampling is to estimate some measure in the larger population. • Probability sampling: A selection of elements in a population, such that every element has a known, non-zero probability of being selected.* * Last, Dictionary of Epidemiology

  17. Sampling methods - definitions • Sampling universe • Sampling frame • Sampling unit • Basic sampling unit or elementary unit • Sampling fraction • Respondent • Survey subject • Unit of analysis

  18. Sampling methods – simple and systematic sampling • Simple random sampling • Most basic type of sampling • Statistical theory based on SRS • Calculate p values and confidence limits • Output from most statistics computer programs assume SRS • Selection of units is random and independent • Systematic random sampling • Similar to simple random sampling • First sampling unit chosen randomly • Systematic selection of subsequent units • Statistics same as simple random sampling

  19. Sampling methods – simple and systematic sampling Both simple and systematic random sampling require a complete list of all sampling units in sampling universe OR that sampling units be organized to allow systematic selection

  20. Sampling methods – list of sampling units

  21. Sampling methods – organization of sampling units; Stankovic II camp, Skopje, Macedonia - 1999

  22. Sampling methods – organization of sampling units; Cegrane camp, Cegrane, Macedonia - 1999

  23. Sampling methods – organization of sampling units; Chaman camp, Quetta, Pakistan - 2001

  24. Sampling methods – organization of sampling units; Neprostino camp, Tetovo, Macedonia - 1999

  25. Sampling methods – organization of sampling units; Cegrane camp, Cegrane, Macedonia - 1999

  26. Sampling methods – cluster sampling Cluster sampling is probability sampling in which sampling units at some point in the selection process are collections, or clusters, of population elements* * Kalsbeek, Introduction to survey sampling

  27. Sampling methods – cluster sampling • Is almost always multistage • Randomly choose geographic areas as primary sampling units • Selection is probability proportional to size • Final stage of sampling – choose basic sampling unite (For example, households or children)

  28. Advantages Cheaper - basic sampling units closer together Does not need complete list of basic sampling units Disadvantages Decreased precision of estimate Calculation of p values and confidence limits more complicated Sampling methods – cluster sampling

  29. Sampling methods – cluster sampling Stage 1 Stage 2 Stage 3

  30. District: A B D F C E PPS 231 912 3,099 376 484 763 Not PPS 231 912 3,099 376 484 763 Sampling methods – cluster sampling Sampling probability proportional to size

  31. Sampling methods – cluster sampling

  32. Sampling methods – cluster sampling

  33. Cluster sampling – probability of selection 30 clusters of 15 households each Probability of selecting subdistrict within selected district Probability of selecting HH within selected subdistrict Total probability of selecting any HH in country Probability of selecting district 30 x Number of HHs in district ——————— Number of HHs in country Number of HHs in subdistrict ——————— Number of HHs in district 30 x 15 —————— Number of HHs in country 15 —————— Number of HHs in subdistrict x x =

  34. Sampling methods – recommendations • Choose sampling method appropriate to availability and organization of data on population • Use simple or systematic random sampling when possible if logistic savings from cluster sampling not important • Most refugee camps are small enough to so that cluster sampling offers little logistic advantage • For cluster sampling, always sample probability proportional to size in all but the last sampling stage

  35. Final sampling stage

  36. Final sampling stage – EPI method EPI (bottle spinning) method of selecting households

  37. Final sampling stage – EPI method • Simple • Widely known • Easy to train • Results in biased sample

  38. Final sampling stage – selecting households EPI (bottle spinning) method of selecting households 10 9 5 3 4 7 8 6 2 1

  39. Final sampling stage – selecting households EPI (bottle spinning) method of selecting households 1 2 3

  40. Final sampling stage – selecting households EPI (bottle spinning) method of selecting households 10 9 5 3 4 7 8 6 2 1

  41. Final sampling stage – alternate methods • Less simple • Not as widely known • Requires somewhat more training • Results in less-biased sample

  42. Final sampling stage – selecting households Map and dart throw method of selecting households

  43. Final sampling stage – selecting households Segmentation method of selecting households 1 2 3 6 4 5

  44. Final sampling stage – selecting households Simple or systematic random selection of households 27 5 1 22 6 2 17 28 23 11 18 7 12 24 19 13 8 15 25 3 14 20 9 16 29 4 26 30 21 10

  45. Final sampling stage - recommendations • Do not use EPI method • Continue sampling stages until reach small enough area to do alternate method • Map and segment or list, then do simple or systematic random sampling

  46. Sample size

  47. Sample size To estimate sample size for single survey using simple or systematic random sampling, need to know: • Estimate of the prevalence of the outcome • Precision desired • Design effect • Size of total population • Level of confidence (always use 95%)

  48. Sample size

  49. Sample size

  50. Sample size

More Related