450 likes | 456 Views
Learn why program evaluation is important in youth development, its benefits, and available resources. Understand the different types of evaluation and how to implement them effectively.
E N D
Why evaluate? Benefits of program evaluation
YD as EVAL: Webinar Agenda • Why evaluate? • So what is evaluation? • How does evaluation fit youth development? • How can evaluation benefit a YD program? • What resources are available to learn about evaluation?
Benefits of Informal Evaluation • Building relationships • Gathering facts • Offering support, recognition • Troubleshooting problems • Offering guidance
Benefits of Formal EvaluationCompleting Organizational Evaluation • Targeted outcomes and tools • Efficient reporting process • Data for funding proposals • Feedback for improving programs
Who needs to know? Stakeholders Individuals and groups with interest in a program
Rethinking EvaluationRevitalizing Programs …in a user-friendly way… …and put the answers to good use… Ask the right questions…
So what is Evaluation? Evaluation Critical examination of a program’s resources, activities, and outcomes to judge effectiveness, improve quality and results, and inform decisions. (Patton, 1997)
Evaluation Activities • Assessment: Investigating needs and assets • Monitoring: Tracking processes and progress • Evaluation: Documenting quality and outcomes
Merit: Program Quality and Potential Merit: Quality, Best Practices, potential to promote outcomes • Overall: Traits that promote positive development (safety, support, skill-building, self-efficacy… • Particular: Afterschool, Camps, Science, Horsemanship • Formative Evaluation: Is the program working as planned? • Summative Evaluation: What program components contribute to positive outcomes
Worth: Program Outcome and Impact Worth Value of a program, grounded in outcomes that accrue to targeted units (individuals, groups, organizations) • Short-term: Knowledge, Attitudes, Skills, Aspirations • Mid-term: Practices, Knowledge Application, Behaviors • Long-term: Sustained impact of the program on Social, Economic, and Environmental conditions
Significance: Importance and priority Significance: Importance of change to program partners • Overall: Importance to a broad range of stakeholders • Particular: Importance to a particular individual or unit
Evaluation Cycle Preparation • Planning--Setting goals, matching strategies • Focusing--Refining plans, identifying sources • Designing--Developing strategies for collecting data • Selecting Methods--Identifying tools to measure for goals
Evaluation Cycle Implementation • Collecting Data—gathering info from diverse sources • Analyzing Data—reviewing and interpreting data
Evaluation Cycle Utilization • Communicating and Using Results—Sharing data about quality and impact; applying findings to improvement
Positive Youth Development and Evaluation(Formal and Informal) • Life skill and life-blood of program management • Goal Setting/Project Management • Communication • Problem Solving
Positive Youth Development and Evaluation (Formal and Informal) • Life skill and life-blood of program management • Goal Setting/Project Management • Communication • Problem Solving • Key strategy in positive youth development • Experiential Learning = Action & Reflection
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 • Processes: Child Development knowledge base • Principles: Program Theory and Philosophy • Practices: Strategies for Programming …and Evaluation • Products: Results of PYD Programming
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 Processes: Child Development knowledge base Dynamics Maturation Interaction Adaptation Domains Physical Cognitive Social-Emo Stages Early Child Middle Child Adolescence
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 Principles: Program theory and philosophy Developmentally Appropriate Strengths-based, Risk-aware Universal and Inclusive Intentional and Iterative Holistic and Communal Ongoing in Engagement Theories of Change Integrated across Ecosystems Theories of Action
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 Practices: Strategies for program implementation • Experiential • Learning • Project-based • Inquiry-based • Problem-based • Adventure-based • Participatory • Service-learning active listening buddy system Safety Support Skill-Opportunities Self-efficacy-building Quality Practices Social Norms Social Inclusion Structure Synergy choices levels team work Youth-led/ Y-A partnership
YD as EVAL: Authentic Assessmentbased on Hamilton, Hamilton, & Pittman, 2004 Practices: Strategies for program implementation …and evaluation Skills card Check-in/Obs Skills card Expert obs/intx Safety Support Skill-Opportunities Self-efficacy-building Quality Practices Social Norms Social Inclusion Structure Synergy buddy system Photo evidence choices levels active listening team work Skills card Expert obs/intx Rubric Trained obs/intx Demo/ Confirmation Physical evidence
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 Products: Outcomes of programs Character Competence Confidence Connection Caring Contribution
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 Processes: Child Development knowledge base Dynamics Maturation Interaction Adaptation Domains Physical Cognitive Social-Emo Stages Early Child Middle Child Adolescence
Youth Development as Evaluationbased on Hamilton, Hamilton, & Pittman, 2004 Principles: Program theory and philosophy Developmentally Appropriate Strengths-based, Risk-aware Universal and Inclusive Intentional and Iterative Holistic and Communal Ongoing in Engagement Theories of Change Integrated across Ecosystems Theories of Action
YD as EVAL: Minimum Effort Focus Integrate Document Outdoor Skills Cabin Group Life Skills Frequency Intensity Duration Program Quality
YD as EVAL: Maximum Effort Outdoor Skills Cabin Group Life Skills Agent Focus Groups Registration Parent Survey Program Quality
Benefits of Evaluation • Systematic planning, implementation • Monitoring risks, opportunities • Accountability and reporting • Improving and expanding quality • Sustaining, growing funding • Support and recognition of partners • Recruiting, training new partners • Sharing with peers • Building broader support, insight
Barriers to Evaluation “We’re too busy… “Evaluation can only hurt us… “We just don’t have the goods…. “We don’t know what to evaluate…
Resources for Evaluation https://youthdevelopment.ces.ncsu.edu/ E-Basic Basics Factsheets
A difference that counts How evaluation makes a fairer fair… • Safety…Support…Skill-building • Typical/Exceptional care for animals • Problem solving about projects, careers • Individual resilience and group teamwork • Connectedness • Competence • Confidence
So…Why evaluate? • Satisfy our own curiosity… • Track quality and impact… • Use results to improve, show impact, inform partners, and invite new participants • Make the most of our method: action and reflection, learning by doing
Youth Development as Evaluation: Application Outdoor Skills Cabin Group Life Skills Safety Support Skill-Opps Self-efficacy-building Social Norms Social Inclusion Structure Synergy Registration Parent Survey Program Quality Program Quality
“What I do doesn’t make that much difference…” A Fair Appraisal • How did the volunteer promote youth development best practices (safety, support, skill-building, etc.)? • How would these qualities over time promote connectedness, competence, confidence, character, caring, and contribution?
Behold the Benefits Professional Level Formal Evaluation Functions • Reporting achievements to sponsors and decision-makers • Sustaining, improving upon, or enhancing program activities
Eval benefits at the Professional Level(Formal evaluation) • Accountability to program stakeholders • Improvement efforts with staff and volunteers