1 / 13

Pt 1. Tips and Tricks of Performance Reporting

Pt 1. Tips and Tricks of Performance Reporting. Reed Early Oct 2012 CESO-SACO . Lessons learned from masters. Start with understanding of the program logic Establish many indicators, & track few at a time Measure the indicator, but also the cause of if

howie
Download Presentation

Pt 1. Tips and Tricks of Performance Reporting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pt 1. Tips and Tricks of Performance Reporting Reed Early Oct 2012 CESO-SACO

  2. Lessons learned from masters • Start with understanding of the program logic • Establish many indicators, & track few at a time • Measure the indicator, but also the cause of if • Avoid “mad outcome” disease neglecting all else • Do not let the indicator become the mission • Do not abuse indicator i.e. teach to the test (unless it improves the desired long term outcome) • Use indicators to improve program impact, not change program to improve the indicator (as in high stakes testing (false + )

  3. Lessons learned con’t • When outcomes are long term don’t cut funding to programs if none show up immediately (as summative evaluation would do) • Look for ways to improve based on indicators (formative eval ) • In some cases the need justifies the program, whether impacts can be achieved or linked to program • Sometimes important outcomes will never be measureable directly so look for a proxy

  4. Performance Indicators: • ...are the best we can do to measure the outcome • ...can be conceptual but must be measurable (i.e. through a proxy) • ...should not become the total focus instead of the program • ...should be valid and logical indicators • ...must be pragmatic not opportunistic • ...need to be S.M.A.R.T. • ...are the way to clearly and accurately report to audience • ... improves transparency if used with Performance Reporting

  5. Four Lessons for Indicators (Michael Scriven) • Try to get good indicators – and refine those you do have • Watch the organization for indicator abuse, mis-use, or corruption (cart leading the horse) • Watch the indicator for wear – shelf life may be short • Enrich the indicators with discussion of the limitations

  6. Good and bad indicators Performance of a light bulb:  Criterion –by definition (lumens)  Logical - able to read by light  Outcome - durability (yrs)  Output/Process- manufacturer  Input - energy used (watts)  Fashion– is it “cool”  False - cost ($)

  7. Good and bad indicators Performance of an employment program:  Criterion – by definition (#employed)  Logical (# hired)  Outcome (# graduates)  Output/Process- (#enrolled, #seats offered)  Input - ($ of program)  Popularity - Public perception

  8. On money (Michael Scriven) • Rarely should saving $ be the most important outcome • Instead effectiveness should be the goal in order to save $ or increase impact • Positive results often result from increased efficiency • Efficiency = saved $ and increased outcome

  9. For fun - 10 Commandments (Michael Q Patten) • Thou shall have no other gods before performance reporting • Shall have no idols or holy images besides • Shall not take name of performance reporting in vain • Six days thou shall toil but on the 7th day thou shall report • Honour thy public, stakeholders and data providers • Do not kill the messenger bearing poor results

  10. Fun continued ...  • Thou shall not commit adultery with stakeholders or data providers or anyone performance depends on • Thou hall not steal work of other reports • Thou shall not bear false witness – i.e. use bad data or make wrong conclusions • Thou shall not covet thy stakeholder’s data, or results or anything covetous (Michael Q Patten)

  11. Performance Reporting • Principles apply to all government • Apply to performance reports • Can be measured against standards • Below standard • At standard • Above standard • Exemplary

  12. Performance Reporting Principles 1. Explain the public purpose served 2. Link goals and results 3. Focus on the few, critical aspects of performance 4. Relate results to risk and capacity 5. Link resources, strategies and results 6. Provide comparative information 7. Present credible information, fairly interpreted 8. Disclose the basis for key reporting Judgements

  13. For more information consult Performance Reporting Principles For the British Columbia Public Sector http://www.bcauditor.com/pubs/2003/special/performance-reporting-principles-british-columbia-bc • end

More Related