Research Methodologies: an introduction

Research Methodologies:an introduction CEGAA-OSI EE Partners Workshop March 09 Urbanus Kioko Centre for Economic Governance and AIDS in Africa

Your Understanding of Research?

What is research? • Systematic means of solving a problem • Process in which observable verifiable data is systematically collected in order to describe, design or predict events • Systematic, objective and scientific investigation of a phenomenon targeting specific aspect for the purposes of discovering, interpreting facts, principles and theories • The search for knowledge through objective and systematic methods of generalisation and theory formulation • To inform decision making by producing evidence

Evidence-Based Advocacy Framework

What is your desired long-term impact, the outcome of your efforts, the rights you wish to protect or promote, constitutional and legislative obligations, international/regional commitments. What are you hoping to achieve? SMART objectives to bring about the change needed to achieve your GOAL. (SMART: Specific, Measurable, Achievable, Realistic, Time-bound), (Note that the advocacy strategy should begin to be developed now, but will be finalized when the preliminary research findings are available). Advocacy objectives Project definition/overall goal Clearly state what the problem is that you are going to address. Many sources to find topics or issues that can lead to research questions: Personal experience, Articles in professional periodicals, Unpublished research by others. Focus your research so that it is "do-able." Be careful! Don't try to do too much in one study. Problem statement Key Stages in Research:1. Identifying the problem

Criteria for prioritising problems for advocacy research • Relevance in terms of being a priority problem • How large or widespread is the problem • Who is affected • How severe is the problem • Avoid duplication • Feasibility • Political acceptability • Applicability of possible results and recommendations • Urgency of the results for making a decision • Ethical acceptability

What do you want to answer? What principles do you want to test? What data will the stakeholders require for the advocacy messages? Use Lit.Review (throughout process) State, private, donor, household expenditure/ allocations? Sources, agencies, providers, services? Inputs/ outputs/ outcomes/ impact? Federal/national and/or state/sub-national? Recurrent and/or development? Sectoral analysis? Programme analysis? Years to be covered? Population focus? Vulnerable group/ gendered perspective? Your framework of analysis – rights-based, sectoral, issue, equity? Defining the specific research question/s The scope of the research To achieve the advocacy goal, identify more realistic aims of the research required to support the advocacy campaign, and develop specific, measurable objectives of the RESEARCH. Research aim and objectives Step 2: Defining Research Aims and Objectives

Survey of professional literature that is pertinent to your particular question. • Find what others have done in relation to topic of interest. • Why review • Prevent duplication • Assist in refining statement of problems • Acquaint researcher with methodologies already used • Assist in justifying a research • What potential sources of data • Individuals, groups, and organisations • Published information, unpublished information • Where can information be found • Local • National • International e.g. internet The scope of the research-Literature review Step 2: Defining Research Aims and Objectives

Sources of data? Collectors of data? Users of data! Benefits/ risks. Plan to involve all the stakeholders to ensure ownership and relevance of data collected and research process. Quantitative, qualitative, participatory, costing, economics – micro/macro analysis, budgetary analysis, expenditure tracking techniques, community/Citizen action research – social audits, satisfaction surveys, score cards. Donor surveys, household/ clinic/ district surveys. Focus Gp Discs. Key partners & participation process. Stakeholder involvement/ ownership. Methodology: Study design Identification of appropriate indicators What indicators/ information will you use to show the principles you are trying to measure? Steps 3: Indicators, Methods

Research Design… • Research design can be thought of as the structure of research -- it is the "glue" that holds all of the elements in a research project together • A design is used to structure the research, to show how all of the major parts of the research project -- the samples or groups, measures, treatments or programs, and methods of assignment -- work together to try to address the central research questions.

Types of Research Design • Primary vs Secondary • Exploratory vs Conclusive • Quantitative (BMET analysis, pre-coded closed ended questionnaires, surveys) and/or • Qualitative (open ended questions) • qualitative research attempts to capture "the big picture" and see how a multitude of variables work together in the real world. • It examines naturally occurring behavior, so the investigative methods are as non-intrusive as possible • Focus group discussions) • Continuous (panels, tracking studies) vs Ad hoc / once-off (surveys)

Primary Research Secondary Research Data generated specifically to address the problem/ issue Data originally generated for another reason Primary vs Secondary

Representative sample? Apply formula. Purposive sampling of districts/ households? Give criteria for respondents’ inclusion/ selection. How will you identify and select respondents from the group you are focusing on? Budget analysis, expenditure analysis, literature review, focus groups, workshops, interviews (face-to-face, telephonic/ questionnaires (self-administered?), surveys, roundtable discussions, stakeholder meetings, observation? Tools Sampling techniques Link to earlier decision on population/ vulnerable group. Identify key ‘holders’ of information (govt, donor, service users). Regions/ states? Districts? Households? Individuals? Key officials from MoF, MoH, Parliament,, donors, programme managers, relevant civil society groups, NGOs, etc. Key respondents / your sample population Steps 4: Sampling

Sampling techniques • The listing of the accessible population from which you'll draw your sample is called the sampling frame • Sampling: process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen • The sample is the group of people who you select to be in your study.

Random sampling • Random sampling-gives each of the units in the population targeted a calculable and non-zero probability of being selected • Assumption is that each unit has equal chance of being selected • SRS: the members of the population (N) of interest are numbered and a number (n) of them are selected using random numbers without replacing them • Each sample unit can only appear once in the sample • Each pop member has an equal chance of selection

Systematic random sampling • Steps you need to follow in order to achieve a systematic random sample: • number the units in the population from 1 to N • decide on the n (sample size) that you want or need k = N/n = the interval size • randomly select an integer between 1 to k • then take every kth unit N = the number of cases in the sampling frame • Note: • n = the number of cases in the sample • NCn = the number of combinations (subsets) of n from N • f = n/N = the sampling fraction

Systematic random sampling… • Let's assume that we have a population that only has N=100 people in it and that you want to take a sample of n=20. • To use systematic sampling, the population must be listed in a random order. • The sampling fraction would be f = 20/100 = 20%. • Interval size, k, is equal to N/n = 100/20 = 5. • Now, select a random integer from 1 to 5. • In our example, imagine that you chose 4. • Now, to select the sample, start with the 4th unit in the list and take every k-th unit (every 5th, because k=5). • You would be sampling units 4, 9, 14, 19, and so on to 100 and you would wind up with 20 units in your sample.

Stratified random sampling • Sometimes called proportional or quota random sampling, • Involves dividing your population into homogeneous subgroups and then taking a simple random sample in each subgroup – so that each sub-group is proportionally represented in the sample – you can then make inferences about these groups in the population. • Objective: Divide the population into non-overlapping groups (i.e., strata) N1, N2, N3, ... Ni, such that N1 + N2 + N3 + ... + Ni = N. • Then do a simple random sample of f = n/N in each strata.

Cluster (Area) Random Sampling In cluster sampling, we follow these steps: • divide population into clusters (usually along geographic boundaries) • randomly sample clusters • measure all units within sampled clusters

Multi-stage sampling • The selection of clusters can be multi-stage • Involves selecting districts, the primary sampling units, within a region for the sample, and within these sample electoral wards and finally within these a sample of households.

Purposive sampling • Is a deliberate non-random method sampling, which aims to sample a group of people, with particular characteristic. • The criteria for selection must be justified and relate to the questions you are trying to answer • Can be justified with difficult access to certain populations eg. HIV+ patients attending the ARV clinic • Can be justified based on costs and time eg. Small case study • Is not generalisable!!

Sources of data Budget books, budget reviews, MTEF plans, expenditure reports, Auditor General reports, ministry budgets and annual reports and costing reports, strategic plans, MoF reports, Parliamentary reports especially of budgeting committees, Public Expenditure Reports, health information systems, legislation governing public financing, Reports of the implementing agencies of the strategic plan, central bureau of statistics, household data, census, surveys (economic eg. I&E, health), previous studies, programme evaluations, impact assessments, any costing exercises (against which to measure the adequacy of budget allocations), financial management agencies, donor reports, etc Step 5: Data source selection

Stakeholder meetings to present preliminary findings, to discuss and influence the report recommendations. Country and external experts to review draft report – experts in health issue, public finance, budget systems etc. Internal review process within the team – sharing of experiences and solutions, editing each other’s reports. Using civil society networks to give critical feedback. h. Review process – to ensure validity, accuracy and reliability of data Plan for using the tools to collect the information. Means of capturing data – software programmes, research assistants, data capturers. • Questionnaires – closed/open ended, • Questionnaires (mailed or in-person) • Observations • Interviews-personal: interviewer works directly with the respondent, can probe/ask follow up qns g. Data collection & management process Data collection tools Step 6: Data Collection

Data collection tools-questionnaire design-some tips • Question Content • How well does the qn address the content you are trying to get at. • Is the Question Necessary/Useful? • Are several questions needed? double-barreled question—need to split e.g. what is your occupation and monthly income? • Does the qn have the necessary information for the respondent to be able to answer it? • Specific-too general making information we obtain more difficult to interpret. • E.g. How well did you like this workshop? • on some scale ranging from "Not At All" to "Extremely Well.“ What does it mean to say you liked the workshop very well? Biased or Loaded question: your own biases and blind-spots may affect the wording e.g. you may be against random street drug testing. What do you see as the drawbacks of random street drug testing? • Is the wording too direct? • too direct/disturbing for respondents. • How did you feel about this workshop? Step 6: Data Collection…

Some more tips for Consideration • Conventions or rules-of-thumb in the survey design. • Start with easy, non-threatening questions, put more difficult, qns near end • Never start a mail questionnaire with an open-ended question • For filter or contingency questions, make a flowchart

Project planning & management • Fully brief the team – project manager, executives • Key questions to be answered • Objectives and methodology • Sampling/ selection details • Quality control procedures • Timelines • Costs for each stage • Human resource requirements TIP: Take time, keep records. Successful projects are well communicated, planned and every team member is clear about their role Step 7: Project Management

Select and train interviewers very carefully - match interviewer profile with • interviewee profile? • - sensitive subjects • - language/ cultural issues • Document the quality control procedures • Train on field research ethics and specific project requirements • TIP: Quality control all stages of fieldwork every day through the Supervisor network Select, train & brief field team Pilot • Carry out a pilot exercise • - check flow of questions, skip patterns, • - language of questions, selection procedure, • length of interview • Conduct a pilot FGD • TIP: Use experienced interviewers/ moderators for good quality feedback Managing Data collection process

Develop editing rules and checks for all questionnaires • - Check frequency distributions • - Check against known statistics • - Look out for inconsistencies • TIP: Apply common sense. ‘Interesting’ data are often wrong • Excel, Quantitative • - develop analysis specification (SPSS) • - use techniques to help you • understand relationships in the data • Qualitative (NVIRO) - develop template for verbatim transcripts • - common themes and differences • TIP: Think through the analysis as you design the instruments “How will I • analyse this?” Data entry, cleaning and checking Data analysis Step 8: Data processing & analysis

Revisit objectives • Triangulate with other sources • Verify prelim findings with key stakeholders • Compare with previous work (lit.review) • Contextualise • Find the story Data interpretation Step 9: Interpreting & Verifiying the findings

Plan and structure to tell the story – using • the data as supporting evidence • Summarise the key findings • Keep it short (use appendices) • Illustrate with diagrams, charts, • illustrative quotations • Dissemination workshop • Policy briefs, media etc • TIP: Rehearse presentations, circulate • reports internally for comment. The • quality of the deliverable is the basis of the policy maker’s/govt’s assessment of your work Report writing and presentation Step 10: Reporting & Dissemination

Closing remarks • Research does not only inform advocacy, Advocacy also makes demands of research • Research needs to answer key questions in order to be useful for advocacy purposes

Thank You For more information contact: • Urbanus Kioko • Centre for Economic Governance and AIDS in Africa • Email:urbanus@cegaa.org • Tel: +254-720-209-100

Research Methodologies: an introduction