Dr. Gary J. Skolits Tiffany L. Smith Institute for Assessment and Evaluation 865-974-2777

1. Dr. Gary J. Skolits Tiffany L. Smith Institute for Assessment and Evaluation 865-974-2777 gskolits@utk.edu Evaluation Perspectives:Logic Models, Impacts and Budgeting 2011 Systems ScienceGrantsmanship WorkshopUSDA National Institute of Foodand AgricultureAugust 9, 2011

2. EvaluationPresentation Topics • Logic model basics • Documenting project impacts • Evaluation do’s and don’ts • Evaluation budgeting perspectives

3. Logic Model Basics

4. Logic Model Basics • A logic model is a visual depiction of how a project intervention is expected to produce a desired outcome

5. General Logic of a Project Social Need (Problem) Project Intervention (Action) Results (Change)

6. Logic Model Drivers

7. Logic Model Elements • Needs • Purpose • Inputs • Outputs • Outcomes

8. Needs • What problems deserve attention? • Always competing needs • Select critical need to address • Define the need

9. Purpose • What do you seek to accomplish towards addressing this problem? • Select aspects of the problem to address • Define the specific purpose of your project

10. Inputs • What resources will you invest? • Money • People • Materials • Equipment • Partners

11. Outputs • What will you do? • Services • Training • Products • Whom will you reach? • Clients (must define this group) • Typically multiple stakeholders

12. E. Outcomes What results will you achieve? • Short-term (initial impact on participants) • Medium-term (ongoing impact on participants) • Long-term (impact on the problem)

13. 2. Documenting Project Impacts

14. Outcomes – Short-Term • Feedback • Awareness • Commitment

15. Outcomes – Medium-Term • Knowledge/Skills • Disposition • Behavior

16. Outcomes - Long-Term • Achievement of project purpose • Favorable change in the initial need of concern

17. Impact: A Theory of Change • Short -term: If participant reaction is positive, then you theorize that participants are likely to learn desired knowledge, skills, and attitudes. • Medium-term: If participant learning/growth occurs (i.e., knowledge, skills, and dispositions), then you theorize that participants will change their behavior. • Long-term: If behavior changes, then you theorize that there will be a positive change in terms of the initial need.

18. Logic Model Template University of Wisconsin-Madison Extension

19. PLANNING: start with the end in mind What do you want to know? How will you know it? EVALUATION: check and verify

20. 3. Evaluation Do’s and Don’ts

21. A Few Evaluation Do’s • Talk with colleagues sponsoring similar projects for leads on good evaluators • Engage an evaluator with a good reputation and experience with your type of evaluation need • Engage an evaluator early in project planning • Understand the role of formative and summative project evaluation • Recognize the importance of efficient, reliable data collection • Build evaluation data collection into project operations (when possible)

22. A Few Evaluation Dont’s • Be intimidated by evaluation – it is meant to enhance your project • Fail to communicate with your evaluator on a regularly scheduled basis • Forget to address IRB concerns – protect yourself and your clients/stakeholders • Miss the opportunity to use project and evaluation data to add to the literature in your field

23. 4 . Evaluation Budgeting Perspectives

24. 4. Budgeting for Evaluation Some factors to consider • Evaluation costs depend on the required effort of the evaluator • A general rule: 5% to 8% of project budget • Be prepared to negotiate with your evaluator to minimize evaluation costs

25. Budgeting for Evaluation • The more data project staff collect, the lower the evaluation costs. (However, some data should be collected by the evaluator.) • Link any proposed evaluation budget to specific evaluation plan and tasks. • Stay on top of the deliverables promised by your evaluator.

26. Two Key Resources • University of Wisconsin-Madison (Extension)http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html • Kellogg Foundation Logic Model Guide http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx

27. Gary Skolits • gskolits@utk.edu • 865-974-2777 Institute for Assessment and Evaluation University of Tennessee, Knoxville 1122 Volunteer Boulevard; A503 Bailey Education Complex Knoxville, TN 37996 • http://web.utk.edu/~iae/