Developing Performance Measures

1. Developing Performance Measures

2. Purpose of Workshop • To assist you in developing • Performance measures • Measurement methods • To discuss Institutional Effectiveness and changes in reporting requirements OEAS--IE Training Workshop

3. First Validate Your Mission, Goals, and Objectives Why? Necessary and Sufficient How? OEAS--IE Training Workshop

4. Performance Measure • A means of objectively assessing programs, products, activities, or services • Should be related to your mission and goals • Should indicate how you will measure your objective • Should indicate when your objective will be measured • Should indicate who will do the measurement OEAS--IE Training Workshop

5. Structure of a Performance Measure Statement • The (description of the measure) based on (approach used to develop the measure) • The (number of students attending workshops) based on (annual attendance records) • The (satisfaction level of students with the given service) based on (an annual survey of all graduating seniors conducted by OEAS) • The (application processing time for applications) based on (the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft) OEAS--IE Training Workshop

6. First Approach Describe mission Describe vision List goals List objectives Determine at least one performance measure for each objective Second Approach Describe mission Describe vision List goals Brainstorm performance measures Select “preferred” performance measures Set targets which become the objectives Two Approaches to Generating Objectives and Performance Measures OEAS--IE Training Workshop

7. Generating Objectives then Performance Measures • Example: (Admissions) • Goal: To provide timely response to undergraduate applicants • Objective: To reduce the application processing time to less than 14 days for 95% of applications • Performance measure: The application processing time for applications based on the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft OEAS--IE Training Workshop

8. Generating Performance Measures then Objectives • Example: (Admissions) • Goal: To provide timely response to undergraduate applicants • Related performance measures • The number of days to process application based on… • The average number of applications processed per file manager per day based on… • The student satisfaction with the timeliness of the admission decision based on… OEAS--IE Training Workshop

9. Evaluating the Quality of Your Performance Measures • Does each measure relate to the associated mission, goal, and objective? • Is each measure important to management? • Is it possible to collect accurate and reliable data for each performance measure? • Taken together, do the measures accurately reflect the key results of the program, activity or service? • Is there more than one measure for each goal or objective? OEAS--IE Training Workshop

10. Generating Performance Measures then Objectives • Goal: To provide timely response to undergraduate applicants • Objective 1: to increase the productivity of the file managers by 10% • PM: Average number of applications processed per file manager per day • Objective 2: To reduce the application processing time to less than 14 days for 95% of applications • PM: Number of days to process each application OEAS--IE Training Workshop

11. Two Ways of Measuring Performance • Direct measures of performance • Time • Error rates • Compliance • Cost • Number of outputs per input • Standardized tests • Indirect measures of performance • Perceived time • Perceived efficiency • Perceived quality OEAS--IE Training Workshop

12. Example: To reduce the application processing time to less than 14 days for 95% of applications • Direct measure: The application processing time for applications based on the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft • Indirect measure: The perceived time based on survey of a random sample of 500 applicants OEAS--IE Training Workshop

13. Categories of Performance Measures • Input measures (e.g., staff time, materials, equipment, resources) are useful in showing resources or effort used to provide services; however does not show effectiveness • You may be a spending a lot of effort doing the wrong things • Output measures (e.g., number of products produced or services provided) are useful in defining program or service; however, does not reveal quality or efficiency • You may be producing or providing a lot of the wrong things inefficiently or with poor quality OEAS--IE Training Workshop

14. Categories of Performance Measures • Outcome measures (e.g., score on standardized test, distance from proposed targets) are useful in showing the impact or benefit of the program or service • Efficiency measures (e.g., cost per unit of output, outputs per unit of input, outputs per unit time) are useful in showing productivity and cost effectiveness • Quality measures (e.g., reliability, accuracy, courtesy, competence, responsiveness) are useful in measuring the effectiveness in meeting customer expectations • Lack of quality can be measured (e.g., error rates) OEAS--IE Training Workshop

15. Example Goal:To provide timely response to undergraduate applicants • Input measure: The number of file managers • Output measure: The average number of applications processed per day • Outcome measure: The application processing time for applications based on the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft • Efficiency measure: The average number of applications processed per file manager per day • Quality measure: The satisfaction of applicants with the application processing time as measured by a survey OEAS--IE Training Workshop

16. Another Example: Developing Performance Measures • The mission of the XYZ Foundation is to promote the social welfare of persons resident or located in the Orlando Metropolitan Region by all appropriate means, including relief of the poor, care of the sick and aged, care and nurture of children, elimination of vermin… OEAS--IE Training Workshop

17. Example Goal, Objective, and Performance Measure • Goal: To minimize the number of rats in the City of Orlando • Objective: To reduce the number of rats to ten per city block in the next year • Performance measure: The number of rats per city block as measured by an extensive search of a random sample of 10 city blocks OEAS--IE Training Workshop

18. PMs Relate to Goal: To minimize the number of rats in the City of Orlando • The dollars spent per rat killed based on Budget Office and exterminator records • Percent increase in the number of traps set this year based on city records • The satisfaction of the residents with the extermination program based on survey of a random sample of residents • The dollars spent on extermination based on the Budget Office records • The number of rats remaining per city block obtained by conducting an extensive search of a random sample of the city blocks in Orlando. • The approximate number of rats killed during the year per city block based on the exterminator reports OEAS--IE Training Workshop

19. Evaluating the Quality of Your Performance Measures • Does each measure clearly relate to the associated mission, goal, and/or objective? • Is each measure important to management? • Is it possible to collect accurate and reliable data for each performance measure? • Taken together, do the measures accurately reflect the key results of the program, activity or service? • Is there more than one measure for each goal and/or objective? • Are your measures primarily outcome, efficiency or quality measures? OEAS--IE Training Workshop

20. Evaluate: Related, Important, Measurable, Comprehensive, and Preferred Type MINIMIZE THE NUMBER OF RATS IN ORLANDO • The dollars spent per rat killed based on Budget Office and exterminator records • Percent increase in the number of traps set this year based on city records • The satisfaction of the residents with the extermination program based on survey of a random sample of residents • The dollars spent on extermination based on the Budget Office records • The number of rats remaining per city block obtained by conducting an extensive search of a random sample of the city blocks in Orlando. • The approximate number of rats killed during the year per city block based on the exterminator reports OEAS--IE Training Workshop

21. Some Resulting Objectives Supporting the Goal MINIMIZE THE NUMBER OF RATS IN ORLANDO • To improve the cost effectiveness by 10% by Fall 2001 • To obtain high to very high satisfaction rating of 90% of city residents • To reduce the number of rats to ten per city block in the next year OEAS--IE Training Workshop

22. Activity: Develop Performance Measures for each Goal • Try to develop an input, output, outcome, efficiency, and quality performance measure for each goal • Identify whether each is a direct or indirect measure • Can you identify a direct measure for the ones that are currently indirect? • Can you identify an indirect measure for the ones that are currently direct? OEAS--IE Training Workshop

23. Develop Objectives • Evaluate quality of performance measure • Related • Important • Measurable • Comprehensive • Preferred Type (outcome, efficiency, quality) • Develop objectives OEAS--IE Training Workshop