1 / 34

Monitoring and Evaluation: Frameworks

Monitoring and Evaluation: Frameworks. Learning Objectives. At the end of the session, participants will be able to: Identify and differentiate between conceptual frameworks, results and logical frameworks, and logic models Design goals and objectives for specific intervention programs

rweaver
Download Presentation

Monitoring and Evaluation: Frameworks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Monitoring and Evaluation:Frameworks

  2. Learning Objectives At the end of the session, participants will be able to: • Identify and differentiate between conceptual frameworks, results and logical frameworks, and logic models • Design goals and objectives for specific intervention programs • Design frameworks for specific intervention programs • Discuss how frameworks are used for M&E planning

  3. Module Outline • Review components of an M&E Plan • Introduction to frameworks • Conceptual Frameworks • Goals and objectives • Results and Logical Frameworks • Logic models • Exercises developing frameworks • Interface between frameworks and M&E planning • Work on group project

  4. Why are frameworks useful? Designing frameworks assists to develop: • Clearly understood program/project goals and measurable, long-term, short-term, and intermediate objectives • Clearly-defined relationships between program/project inputs, processes, outputs, and outcomes, and between program/project activities and the external context (environmental factors) • Sound implementation and M&E plans

  5. Types of Frameworks • Conceptual Frameworks • Results Frameworks • Logical Frameworks • Logic Models

  6. Conceptual Frameworks(a.k.a. research or theoretical frameworks) Diagrams that identify and illustrate the relationships between all relevant systemic, organizational, individual, or other salient factors that may influence program/project operation and the successful achievement of program or project goals. M&E Purpose: • To show where program fits into wider context • To clarify assumptions about causal relationships • To show how program components will operate to influence outcomes • To guide identification of indicators • To guide impact analysis (causal pathways) Similar frameworks: • Proximate Determinants

  7. Individual characteristics Healthstatus Healthypractices Serviceutilization Institutionalcapacity Programsustainability Program supply Technicalinputs Conceptual Frameworks

  8. Conceptual Frameworks National Tuberculosis (TB) Program • External Factors • Demographic: • (age, gender) • Socio-economic:(education, occupation) • Environmental: • (war, weather) TB infection • Health Systems (DOTS) • Availability • Access • Utilization • Case Detection • TB Morbidity • Prevalence • Incidence • HIV co-infection • MDR-TB TBmortality • Program Factors • Political commitment • Donor involvement • National TB program • Co-morbidity • HIV • Malnutrition • Alcoholism • Diabetes • TB Knowledge • Stigma • Treatable • Symptoms Source: MEASURE Evaluation 2004 (draft)

  9. Child malnutrition, death and disability Inadequate Disease dietary intake Insufficient access to food Inadequate maternal & child care practices Poor water/sanitation & inadequate health services Quantity & quality of actual resources - human, economic and organizational – and the way they are controlled Potential Resources: environment, technology, people Conceptual Frameworks Causes of malnutrition in society Outcomes Immediate causes Underlying causes at household/ family level Basic causes at societal level Source: UNICEF, State of the World’s Children, 1998

  10. Activity • Get into groups of 4 or 5 • Develop a conceptual framework for utilization of family planning/maternal and child health services • Identify factors that influence health-service utilization • Arrange factors into a conceptual framework • After 30 minutes, a member of each group will share the framework with all participants

  11. Goals and Objectives • Goal: a broad statement of a desired, long-term outcome of the program • Objectives: statements of desired, specific, realisticand measurable program results • SMART • Specific: identifies concrete events or actions that will take place • Measurable: quantifies the amount of resources, activity, or change to be expended and achieved • Appropriate: logically relates to the overall problem statement and desired effects of the program • Realistic: Provides a realistic dimension that can be achieved with the available resources and plans for implementation • Time-based: specifies a time within which the objective will be achieved Source: GAP 2003

  12. Goals and Objectives National AIDS Committee (NAC) 2005 Strategic Plan • Goal: Prevent the spread of HIV epidemic and minimize its impact on nation by 2009 • Objective 1: Reduction of HIV prevalence in nation • Objective 2:Improved health and quality of life of people infected & affected by HIV/AIDS • Objective 3:Strengthened capacity of NAC and stakeholders to respond to the HIV/AIDS epidemic at all levels through improved research, M&E and improved management and coordination

  13. Child malnutrition, death and disability Inadequate Disease dietary intake Insufficient access to food Inadequate maternal & child care practices Poor water/sanitation & inadequate health services Quantity & quality of actual resources - human, economic and organizational – and the way they are controlled Potential resources: environment, technology, people Goals and Objectives: Activity Outcomes Immediate causes Underlying causes at household/ family level Basic causes at societal level Source: UNICEF, State of the World’s Children, 1998

  14. Results Frameworks Diagrams that identify steps or levels of results and illustrate the causal relationships linking all levels of a program’s objectives Purposes: • Provides a clarified focus on the causal relationships that connect incremental achievement of results to the comprehensive program impact • Clarifies project or program mechanics and factors’ relationships that suggest ways and means of objectively measuring the achievement of desired ends Other terms used: • Strategic frameworks

  15. Results Frameworks Source: David Marsh, 1999

  16. Results Frameworks Tuberculosis (TB) Control Programs SO1: Increase tuberculosis case-detection rate to 70% IR1: Increased availabilityof quality services IR2: Increased demand for quality services IR2.1: Customerknowledge of TB improved IR1.1: Services increased IR1.2: Practitioners’ skillsand knowledge increased IR2.2: Social support for TB practices increased IR1.3: Improved program management Source: MEASURE Evaluation, 2004 (draft)

  17. Results Frameworks Donor/USAID Reproductive Health Program SO1: Increased Utilization of Family Planning/Reproductive Health Services IR1 Strengthened sustainability of FP/RH Program IR2 Expansion of high quality FP/RH services in the public and private sectors IR2.1 Increased availability of postpartum and post-abortion FP services IR1.1 Improved policy environment for the provision of FP/RH services in the public and private sectors IR2.2 Increased accurate knowledgeof clients about modern contraceptive methods and FP services IR1.2 Strengthened NGO advocacy for FP program IR2.3 Improved job performance of health providers, trainers, and administrators Source: USAID/Turkey Performance Monitoring Plan, 1998-2001

  18. Results Frameworks FIVE-YEAR GOAL: Foundation Established for Reducing Famine Vulnerability, Hunger and Poverty SO 14: Human capacity and social resiliency increased IR 14.1: Use of high impact health, family planning, and nutrition services, products, and practices increased IR 14.2: HIV/AIDS prevalence reduced and mitigation of the impact of HIV/AIDS increased IR 14.3: Use of quality primary education services enhanced IR 14.2.1: Reduced risk behavior IR 14.3.1: Community participation in the management and delivery of primary education services strengthened IR 14.1.1: Community support for high impact health interventions increased IR 14.2.2: Mother-to-child transmission of HIV reduced IR 14.1.2: Availability of key health services and products improved IR 14.3.2: Planning, management, and M&E for delivery of primary education services strengthened IR 14.2.3: Access to care and treatment for people living with HIV and AIDS increased IR 14.1.3: Quality of key health services improved IR 14.2.4: Care and support for orphans and vulnerable children expanded IR 14.3.3: Quality of primary education improved IR 14.1.4: Health sector resources and systems improved IR 14.3.4: Equitable primary education services strengthened IR 14.2.5: A more supportive environment for responding to HIV/AIDS Source: USAID/Ethiopia Integrated Strategic Plan, February, 2004 (revised draft)

  19. Logical Frameworks Present a standardized summary of the project and its logic. Purposes: • Summarizes what the project intends to do and how • Summarizes key assumptions • Summarizes outputs and outcomes that will be monitored and evaluated Other terms used: • Logframe matrix

  20. Logical Frameworks

  21. Logical Frameworks Taskforce on Communicable Disease Controlin the Region X: Tuberculosis

  22. Logical Frameworks Taskforce on Communicable Disease Controlin the Barents and Baltic Sea Regions: Tuberculosis

  23. Logical Frameworks Taskforce on Communicable Disease Controlin the Barents and Baltic Sea Regions: Tuberculosis(only one output and related activity shown here)

  24. Activity • Return to small groups from previous activity • Develop goals and objectives for a program aimed at increasing utilization of FP/MCH health services • Develop a results or logical framework for this program • In 45 minutes, a member of each group will share the goals, objectives, and framework

  25. Logic Models Diagrams that identify and illustrate the linear relationships flowing from program inputs, processes, outputs, and outcomes. Inputs or resources affect processes or activities which produce immediate results or outputs, ultimately leading to longer-term or broader results, or outcomes. Purposes: • Provides a streamlined interpretation of planned use of resources and desired ends • Clarifies project/program assumptions about linear relationships between key factors relevant to desired ends Other terms used: • M&E Frameworks, Logical Frameworks

  26. Logic Models: Training INPUT PROCESS OUTPUT OUTCOME IMPACT Develop clinical training curriculum Conduct training events Practitioners trained in new clinical techniques Increase in clients served by (newly) trained providers Declining morbidity levels in target population

  27. Logic Models Voluntary Counseling and Testing (VCT) Problem Statement:HIV infection rates continue to rise, underscoring the importance for people to know their serostatus, develop personalized risk-reduction strategies, and access care and treatment services. OUTPUTS INPUTS PROCESSES OUTCOMES IMPACTS Financial Resources Supervision and training for VCT personnel Clients (HIV+ and -) develop and adhere to personalized HIV risk-reduction Risk behaviors decreased HIV transmission rates decreased People know their HIV status Demand for Services Provide pre-test counseling Increase in care, prevention, and treatment services for HIV+, HIV-, and discordant couples Infrastructure HIV+ Clients develop and adhere to personalized HIV care, support, and treatment plans Provide HIV testing HIV incidence decreased VCT MIS** HIV+ people are referred to appropriate services Provide post-test counseling VCT protocols, guidelines, and training documents** Refer HIV+ clients to appropriate services (PMTCT, care and support, treatment) HIV morbidity & mortality decreased Counseling and Testing Personnel* Health outcomes of HIV + improve Client records are available Continuity of care is available HIV test kits Maintain patient records Program and services are improved Service reports are produced Referral system for prevention & Tx services** Complete reporting requirements

  28. Results Framework Tuberculosis (TB) Control Programs SO1: Increase tuberculosis case detection rate to 70% IR1: Increased availabilityof quality services IR2: Increased demand for quality services IR2.1: Customerknowledge of TB improved IR1.1: Services increased IR1.2: Practitioners’ skillsand knowledge increased IR2.2: Social support for TB practices increased IR1.3: Improved program management Source: MEASURE Evaluation, 2004 (draft)

  29. Logic Model Portion of model for tuberculosis control relating to increasing demand for quality services • IMPACT • Decreased TB infection, morbidity, and mortality • INPUT • Human and financial resources to develop and print an educational brochure • PROCESS • Distribute brochure to health facilities • Meet with physicians to promote distribution of brochure • OUTPUT • Brochure distributed to clients of facilities • OUTCOME • Increased customer knowledge of TB transmission and treatment • Increased demand for quality TB services

  30. Activity • Return to small groups from previous activities • Develop logic models for one of the activities of your program to increase utilization of FP/MCH health services • After 20 minutes, a member of each group will share one of the logical frameworks

  31. Frameworks for M&E Planning • Purposes: • clarifying assumptions, goals, and interrelationships between factors relevant to the project or program • defining objectives • selecting activities • defining levels of performance and desired results in terms of planned activities and realistic, objective impacts • Monitoring and evaluation plans incorporate: • program managers’ assumptions and objectives, in a given context • a schematic design displaying the directional linkages between key program elements and/or planned results, and other relevant factors

  32. Summary of Frameworks

  33. References • AusGuide. The Logical Framework Approach. http://www.ausaid.gov.au/ausguide/ausguidelines/1-1-1.cfm • Bertrand, Jane T., Magnani, Robert J, and Rutenberg, Naomi, 1996. Evaluating Family Planning Programs, with Adaptations for Reproductive Health, Chapel Hill, N.C.: The EVALUATION Project. • Global AIDS Program. (2003) Monitoring and Evaluation Capacity Building for Program Improvement Field Guide, Version 1. U.S. Centers for Disease Control and Prevention, Atlanta, GA. • Marsh, David. 1999. Results Frameworks & Performance Monitoring. A Refresher by David Marsh (ppt) http://www.childsurvival.com/tools/Marsh/sld001.htm • Tsui, Amy. 1998. Frameworks (ppt). Presented at the Summer Institute, University of North Carolina, Chapel Hill. • Tsui, Amy. 1999. Frameworks (ppt). Presented at the Summer Institute, University of North Carolina, Chapel Hill. • UNICEF. 1998. State of the World’s Children.

  34. Group Projects • Form groups for your group projects • For your project, develop • Goals and objectives • Conceptual framework • One other framework: results, logical or logic model

More Related