1 / 24

Programme Evaluation: The Trident

Programme Evaluation: The Trident. Professor Elaine Hogard Director of Assessment & PE Professor of Program Evaluation Northern Ontario School of Medicine Canada. The Trident. The nuts and bolts of Programme Evaluation A “How to” Guide Tried( ent ) Tested. Programme Evaluation: .

christmas
Download Presentation

Programme Evaluation: The Trident

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Programme Evaluation:The Trident Professor Elaine Hogard Director of Assessment & PE Professor of Program Evaluation Northern Ontario School of Medicine Canada

  2. The Trident • The nuts and bolts of Programme Evaluation • A “How to” Guide • Tried(ent) Tested

  3. Programme Evaluation: Is there life beyond consumer feedback? Retrieved from: http://sphotos-b.xx.fbcdn.net/hphotos-frc1/c248.0.403.403/p403x403/65790_570891012929568_1349911854_n.jpg. (Watterson, 1995)

  4. The Trident: A Comprehensive SystematicProgramme Evaluation Framework • Not just consumer views but MSPs • Outcomes and Impact • Process and Program Delivery

  5. The Trident as Framework • Framework for Evaluation Questions • Framework for Data Gathering • Framework for Conclusions • Framework for Recommendations

  6. Overview • Programme Evaluation • Continuous Quality Improvement • Trident Approach • Outcomes • Process • Multiple Stakeholder Perspectives • Aspirations and Frustrations • Successes • References

  7. Why Program Evaluation? • Accountability • Feedback for Practice • Record of Achievement • Data for Research • Dissemination and Publications • Recommendations for Policy and Practice

  8. Continuous Improvement Cycle

  9. Trident What do people think of the programme? Is the programme meeting it’s objectives? How is the programme working? Outcomes Process Multiple Stakeholder Perspectives

  10. Trident Outcomes Logic Model Process Reconstitutive Ethnography MSP Realistic Evaluation

  11. Why the Trident? • Is tried and tested in more than 80 programme evaluations • Structures evaluation through focus on three main questions • Integrates quantitative and qualitative data • Ensures comprehensive data gathering • Facilitates hypothesis testing linking two or three data streams • Is intelligible to contractors, deliverers and recipients

  12. The Trident at Work • Training & Education of Nurses • Sure Start Early Education • Safer Homes • Early Language Development • Shaping the Future through Health Knowledge • Personality Disorder Network • Personalization for Learning Disability • Dementia Support Program • Tourism & Gardens • Village Halls and Carbon Footprint

  13. Outcomes • Programme objectives/outcomes • Logic Model: Short Term ; Medium Term; Long Term Outcomes • Impact • Individual and social change • Measurable • Not Process

  14. Process • Programme delivery • What happened • Process records • Plans • Participant recollections • Reconstitutive ethnography

  15. Multiple Stakeholder Perspectives • Programme recipients • Programme deliverers • Professionals • Volunteers • Managers • ‘Experts by Experience’

  16. Trident Characteristics • Formative and Summative • Qualitative and Quantitative • Partnership: Steering Group • Participatory: Capability Enhancement • No Surprises

  17. Aspirations • Range of Outcomes • Differentiated In-depth Process • 360 degree Stakeholders • Evaluation Questions • Data to answer questions • Linking data streams in hypotheses for example: • Outcomes and delivery method(Outcome and Process) • Recipient approval and outcomes (Stakeholder Perspectives and Outcomes) • Recipient characteristricsand outcomes (Process and Outcomes)

  18. RCTs: The Gold Standard • Randomised controlled trial • Experimental Group: Control Group • Counterfactual impact evaluation • Quasi-experiments • Comparison

  19. Common Frustrations • Outcomes poorly defined • Process descriptions superficial or nonexistent • Stakeholders limited to recipients • Unclear research questions • Poor data gathering • Loss of momentum

  20. Successes • Impact on practice • Impact on policy • Record of achievement • Dissemination of best practice • Programme evaluation; practice and policy

  21. Best Laid Schemes… Retrieved from: www.rightwingnews.com/wp-content/uploads/2011/09/success1.jpg

  22. Evaluation References • Sines D, Hogard E and Ellis R (2012) “Evaluating Quality of Life in Adults with Profound Learning Difficulties Resettled from Hospital to Supported Living in the Community” The Journal of Intellectual Disabilities 16(4), p.247-263. • Hogard Eand Ellis R (2010) “An Evaluation of a Managed Clinical Network for Personality Disorder: Breaking New Ground or Top Dressing?” Evaluation in Clinical Practice 16(6), p.1147-1156. • Hogard E (2007) “Using Consultative Methods to Investigate Professional-Client Interaction as an Aspect of Process Evaluation” American Journal of Evaluation 28(3), p.304-317. • Ellis R and Hogard E (2006) “The Trident: A Three-Pronged Method for Evaluating Clinical, Social and Educational Innovations.” Evaluation 12(3), p. 372-383.

  23. Overview • Program Evaluation • Continuous Improvement • Trident Approach • Outcomes • Process • Multiple Stakeholder Perspectives • Aspirations and Frustrations • References

  24. Thank You !! • For Listening • Any questions/comments • Keep in touch • ehogard@icloud.com

More Related