How to design and implement Equity-focused evaluations

1. Marco Segone, UNICEF Evaluation Office, co-chair UNEG TF on NECD, Co-chair EvalPartners and past Vice President, IOCE Michael Bamberger, independent consultant How to design and implement Equity-focused evaluations

3. The books are available for free atMyM&E Virtual Library: www.mymande.org

5. Outline What does equity mean? Why does equity matter? What are the determinants of inequity? Equity-based programming: what’s the final aim? What is an equity-focused evaluation? How to manage Equity-focused evaluations?

6. What does equity means?

7. Defining Equity • Disparities between population groups • Some of these disparities may be unavoidable (e.g. driven by biology) • The disparities between population groups that are avoidable and unfair are termed inequities • Equity is therefore based on notions of fairness and social justice

8. Why does equity matter?

9. Why equity Equity has a significant positive impact in reducing poverty Equity may have a positive impact on economic growth

10. Why equity Equity has a positive impact in the construction of a socially fair and democratic society Unequal opportunities of social groups in society is often a significant factor behind social unrest, which may lead to crime or even violent conflict, with negative effects for the social cohesion of a society. Prolonged inequity may lead to the “banalization” of inequity Inequity constitutes a violation of human rights and hampers the equitable achievements on Human Development and MDGs

11. What are the determinants of inequity?

12. Determinants: Income Nepal stunting trends and equity Source: DHS data. From NAGA pg. 27

13. Determinants: locations District level Data - Nyanza Province

14. Determinants: locations and Ethnicity In Cambodia, % of 17 years olds who NEVER attended school

15. Determinants: Gender The majority of out-of-school children continue to be girls (53%); achieving gender parity would mean 3.6 million more girls in primary school

16. Socio-economic: Disabilities 90% of children with disabilities in developing countries do not attend school

17. Multiple Dimensions: Who is not learning? Disparity in learning

18. Equity-based programming: what’s the final aim?

19. Final aim • All rights for all children everywhere, by prioritizing the most deprived

20. Reduction of equity gap All target group Reduced equity gap Increased equity gap Most vulnerable within group

21. What is an equity-focused evaluation?

22. What is an equity-focused evaluation? An assessment made of the relevance, effectiveness, efficiency, impact and sustainability of interventions on equitable development results.

23. What’s the impact? Nepal stunting trends and equity Source: DHS data. From NAGA pg. 27

24. Purposes of equity-focused evaluation • Accountability • Organizational learning and improvement • Evidence-based policy advocacy • Contribute to Knowledge Management • Empowerment of worst-off groups National Capacity development for equity-focused M&E systems

25. HOW to manageEquity-focused evaluations?

26. Key Steps 2. Preparing the ToR of an EFE 4. Collecting/ analysing evidence 6. Utilizing the evaluation 1 2 5 6 3 4 5. Findings, conclusion and reccs 1. Preparing for the EFE 3. Designing the evaluation MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS

27. Key Steps 1 2 5 6 3 4 1. Preparing for the EFE MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS

28. . Preparing for the Equity-focused evaluation • Determining the evaluability of the intervention’s equity dimensions • Identifying evaluation stakeholders, including worst-off • Identifying intended use by intended users • Ensuring appropriate budget • Identifying potential challenges in promoting and implementing Equity-focused evaluations

29. Key Steps 2. Preparing the ToR of an EFE 1 2 5 6 3 4 1. Preparing for the EFE

30. Preparing the ToR of an EFE • Defining the scope and purpose of the evaluation • Framing evaluation questions focusing on equity • Selecting technically-strong and culturally-sensitive evaluation team

31. Key Steps 2. Preparing the ToR of an EFE 1 2 5 6 3 4 1. Preparing for the EFE 3. Designing the evaluation

32. Designing the evaluation • Selecting the appropriate evaluation framework • Theory of Change • Bottleneck analysis

33. Bottleneck supply and demand framework: factors affecting use of services by vulnerable populations • Contextual factors • Economic • Political • Institutional • Legal and administrative • Environmental • Supply side factors • Coverage • Budgets and available resources • Demand side factors • Knowledge, Attitudes and Practice of vulnerable groups • Community ownership • Culturally acceptable services • Culturally sensitive staff • distance • cost of travel and fees • available transport • time • poverty Use of services by vulnerable population

34. Designing the evaluation • Selecting the appropriate evaluation framework • Selecting the appropriate evaluation design

35. Selecting the appropriate evaluation design and tools Mix-Methods and triangulation Use for all stages of the evaluation • Contextual factors • Diagnostic studies • Rapid surveys • Secondary data • Key informants • Supply side factors • Project documents • Public expenditure tracking • Surveys • Demand side factors • KAP studies • Case studies • Observation • Key informants • Focus groups • Use of services by vulnerable population • Observation • Surveys • Focus groups • Citizen report cards

36. Designing the evaluation • Selecting the appropriate evaluation framework • Selecting the appropriate evaluation design • Selecting the appropriate evaluation methods

37. Key Steps 2. Preparing the ToR of an EFE 4. Collecting/ analysing evidence 1 2 5 6 3 4 1. Preparing for the EFE 3. Designing the evaluation MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS

38. Selecting the appropriate evaluation methods

39. Menu of tools to evaluate complex equity-focused policies and programmes • Systems approach to evaluation • Unpacking complex policies into components that can more easily be evaluated • Pipeline designs • Policy gap analysis • Using other countries or sectors as the comparison group

40. Key Steps 2. Preparing the ToR of an EFE 4. Collecting/ analysing evidence 1 2 5 6 3 4 5. Findings, conclusion and reccs 1. Preparing for the EFE 3. Designing the evaluation MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS

41. Key Steps 2. Preparing the ToR of an EFE 4. Collecting/ analysing evidence 6. Utilizing the evaluation 1 2 5 6 3 4 5. Findings, conclusion and reccs 1. Preparing for the EFE 3. Designing the evaluation MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS

42. Utilizing the evaluation • Process use • Preparing the evaluation report and alternative forms of reporting • Disseminating the evaluation and preparing a Management Response • Engage the intended users

43. In conclusion: • Focus on understanding how change comes about, not just what happens • Focus on inequities (disaggregated data), contextual factors (socio-economic and cultural context; power relationships) • Recognise complex systems, and use appropriate methods, including Mixed methods • Support worst-off groups to be agents of their own development, not passive beneficiaries

44. Do you want more?

45. Visit the electronic resource center at : www.mymande.org