1 / 13

Measuring Impact

Measuring Impact. v. Kristy Muir Stephen Bennett ENACTUS November 2013. Measuring Impact. Why measure?. Does your or other social enterprises really make a difference? How do you know? How can you prove it? Why bother?

bandele
Download Presentation

Measuring Impact

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Impact v Kristy Muir StephenBennett ENACTUS November 2013

  2. Measuring Impact Why measure? • Does your or other social enterprises really make a difference? • How do you know? How can you prove it? • Why bother? • Purpose of social enterprise: balancing commercial strategy with social environmental or other public benefits • Other benefits from measuring impact • To understand your impact: determine the ‘merit, worth or significance’ of something (Scriven 1967) • To determine the difference between model/enterprise/idea and implementation failure/success • To inform and improve services/ideas/enterprises 2

  3. Measuring Impact Measurement Theories • Definition • "the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of ... Programs” (Rossi and Freeman, 1993) • The systematic collection of information about the activities, characteristics, and outcomes of programs that specific people use to reduce uncertainties, improve effectiveness, and make decisions regarding what those programs are doing and affecting’ (York 2000) [our emphasis] There are a range of ways to measure programs/policies/interventions and there are hierarchies of quality / Mullen et al. (2005) 3

  4. Measuring Impact Measurement Theories • Outcome evaluations study the effects of the program on participants • Process evaluations assess how the program works: establishment and implementation; facilitators and barriers to effective ways of working • Formative and summative evaluation • Formative: help businesses, policy makers and practitioners refine and develop the process through which the program is implemented • Summative: measure the impact of the program • Economic evaluation examples • Social Return on investment • Cost-benefit analysis • Cost-effective analysis Mullen et al. (2005) 4

  5. Measuring Impact Ethics and Morals of Evaluation • Is your approach to measuring impact ethical? • How have you incorporated research ethics – consent and data collection • Impartial evaluator • Operational challenges • Data availability • Attribution 5

  6. Measuring Impact CONTEXT • Social enterprise models • Geographic location • Diverse client demographics • What data is available? • Policy/legislative framework • Social, cultural, political, religious • Comparative programs • Existing evidence of intervention (Coatsworth, 2002, reproduced in Lippman 2004) • What is best approach for collecting evidence based on the contextual factors for your organisation? • What contexts do you need to consider in measuring your impact? 6

  7. Measuring Impact Theory to Practice • Evaluators of complex social programs should: • Develop a theory • E.g. marshmallow test & impulsive behaviour / impulse control • Develop a hypothesis based on that theory (what might work for whom under what circumstances) • If we intervene early to teach children impulse control we can decrease the likelihood they will end up in gaol. Intervention will involve… • Conduct research to test the hypothesis • Make conclusions based on the hypothesis and the findings • Feed back into the development of theory 7

  8. Measuring Impact Logic Frame approach • What? A ‘program logic’ is a systematic, visual representation of the underlying assumptions of a planned program. A results logic illustrates why and how a program is presumed to work. • These models present a sequential, interactive account of how inputs will lead to outputs, which will result in the desired outcomes. • Designing a log frame/theory/hypothesis • What is the major outcome anticipated? Describe the problem your social enterprise is attempting to solve. What is your social mission? • What other goals or sub-outcomes anticipated or desired (which will help work towards the major outcome)? What are the near and long term desired results? • What’s the theory behind why the program will work? What broad systems and structures [or programs] are in place to achieve these outcomes? What are the successful strategies that have helped other organisations achieve the outcomes you desire? What other evidence is available? • What outputs from these systems and structures might contribute to the outcomes? 8

  9. Measuring Impact A Logic Model W.K. Kellog Foundation (2004), Ebrahim& Rangan(2010) • Logic model evaluation method • Map appropriate methods against the program logic • What information could or needs to be collected • Client segmentation • What population level information is available 9

  10. For example…

  11. Measuring Impact What does this mean for me? • What is the business model for your social enterprise? • Purpose • Objectives and strategy & goals • Theory/ research • Finance model • How will you measure the social impact of your social enterprise? 12

  12. Measuring Impact Q&A 13

More Related