Components of good evaluation theory

1. Several Evaluations Theories and MethodsReference: Foundation of Program Evaluation by Sadish, Cook, and Leviton (1991)

2. Components of good evaluation theory Five Components Social Programming: Activities of the social program (Example: Alterna Loan disbursement) Knowledge Construct:How to get the knowledge and understand the knowledge. How the evaluator construct the knowledge. Methodology of the study Valuing Component: What are the values for the evaluator No value judgment (good or bad) Use Component: How evaluators produce results that are useful for social problem solving Utilization of evaluation findings Practice Component: How and when the evaluation should be done What is the purpose of the evaluation. What types of questions should be asked

3. Different Evaluation Theories (Eight) • Policy Research Evaluation (Weiss) • Evaluation for Program Improvement (Wholey) • Theory-Driven Evaluation (Ross) • Responsive Evaluation & Qualitative Method (Stake) • Methodologist Experimenting Society (Campbell) • The Science of Valuing (Scriven) • Functional Evaluation Design (Cronbach) • Participant Oriented Model (Fitzpatrick)

4. Description of Different Evaluation Theories(discuss only three) Policy Research Evaluation (Formative evaluation) • Follow logic model: Explains inputs, implementation, intervening factors, immediate and long-term outcomes • Contributing to improve future programming • Management oriented • Use internal evaluators • Critics • Internal bias to protect him from organizational displeasure • Evaluation suggestions fails to lead to improve the program • Organization resist to change

5. Evaluation for Program Improvement (Summative Evaluation) • Compare actual program performance with standards of expected program performance • Measures successes and failures to meet nation's goals • Draw conclusions about program effectiveness to make decisions • Use external evaluator • Involve multiple stakeholders Critics • Failures to meet nation’s goals • Depends on external funding • Question on evaluator’s objectivity • Controlled by management • Result oriented • Does not confirm evaluation use • Need activities to facilitate evaluation use

6. Theory-Driven Evaluation • Uses program theory as a tool for understanding the program to be evaluated, guiding the evaluation. • Prefers quantitative techniques • Example Community Economic Development (CED) conceptual framework uses to evaluate Alterna micro-loan program • Model guides questions formation & data collection Critics • Skeptical to qualitative method • Needs to choose a conceptual framework that fits best • Work on hypothesis • Comprehensive evaluation, but limited budget, time and other resources hamper the study

7. Participated-Oriented Model • Evaluator himself involved in the program • Evaluator would involve many stakeholders • Use both internal and external evaluators • Involved other stakeholders in the program both in program delivery & in program evaluation. Good model involving people. • Critics • Lengthy process • Difficult to coordinate all stakeholders • Institution has its own priority

8. Activity In the context of your own project, which model do you think works better for you and why? Please discuss with your partner and share your ideas with classmates. • Policy Research Evaluation(Formative) (Weiss) Yes NO • Evaluation for Program Improvement (Wholey) Yes No (Summative) • Theory-Driven Evaluation (Ross) Yes No • Experimental & Quasi-Experimental EvaluationYes No (Campbell) • Others Yes No Thank You