CMP Statewide Evaluation Measurement Plan Summary of Outcomes Workgroup Efforts

1. CMP Statewide Evaluation Measurement PlanSummary of Outcomes Workgroup Efforts Presentation to the State Steering Committee March 15, 2011

2. Goals • Provide background (rationale, legislative goals, objectives) for measurement plan • Review the measurement plan and address questions • Discuss next steps

3. Why we are doing this work • Make stronger statements about the Program’s efforts and successes • Improve the collection of outcome data • Improve service delivery and programs (ability to connect CMP practices with outcomes) • Provide evidence that collaborative management approaches lead to improved child and family outcomes • Promote greater likelihood of ongoing support at the state level

4. Activities to date • Discussed ways to improve measurement processes • Identified CMP core components for measurement • Developed a measurement approach to implement across CMPs • Selected process and outcome indicators in child welfare and juvenile justice areas • Surfaced issues, challenges, solutions

5. Improving measurement: Alignment of interdependent relationships Population Program Model Outcomes

6. Population • The intended core target population of the CMP is children and families who are currently involved, or at risk for involvement, in multiple service agencies

7. Program Models • IOGs • Specified participating service agencies • Includes family advocacy organization • Focus is on establishing ISST • Developing shared processes • ISSTs • Development of a single plan that integrates services across multiple agencies • Family involvement in service planning

8. Outcomes • Key process outcomes • e.g., reduced duplication, cost shifting • Improvements on child and family outcomes • SSC outlined four domains • There are overarching goals • Ensure safety, stability, and well-being for children • Reduce further penetration into systems

9. Minimally, we need to be able to: • Accurately count number of CMP youth and families served by ISSTs • Show that CMP efforts are leading to positive outcomes across the state in at least a few key areas • Measurement plan was developed to meet these minimum requirements • Recognize that this does not capture all of CMPs’ efforts

10. Measurement Plan • Measures will focus on children and families referred to and served by ISSTs • A minimum set of defined process indicators will be uniformly measured across all CMPs • A small set of CW and JJ outcome indicators will be measured on relevant cases • Education and health/mental health outcome indicators will be defined next year

11. Overview of data collection • Process indicators will be collected through individual client-level database • ETO (web-based system) • Or modified local database (provide dataset) • Child welfare and juvenile justice outcomes will be collected from state data systems, where possible • Collected by OMNI at end of fiscal year • Annual reporting will continue in ETO

12. Process Indicators • Number of families referred to and served by ISST and date of ISST meeting • Type of outcome domain each family is enrolled under (CW, JJ, etc.) • Number and type of agencies or systems • Participating in the family’s service planning • Providing services to families • Number of families participating in service planning • Whether or not an integrated treatment plan was developed • Date the case was closed and/or end of service delivery • Outcomes (from selection of common outcomes) that are aligned with case

13. Process indicators: Data collection • Coordinators collect process data from whoever runs ISST meeting • Enter data into ETO sometime after the first ISST occurs • Coordinators will establish processes to: • Locate and record existing client IDs in Trails and Eclipse

14. Outcome Indicators - CW • Number (rate) of CMP youth with new open involvements in Trails after CMP services began • Number (rate) of CMP youth with no substantiated abuse finding after CMP services began • Number of moves that CMP youth experience when in out-of-home placement • Number (rate) of CMP youth discharged to a permanent home

15. Outcome Indicators - JJ • Number (rate) of CMP youth who successfully complete probation and/or parole • Number (rate) of revocations by technical violations where case resulted in unsuccessful termination among CMP youth • Number (rate) of CMP youth who recidivate, resulting in unsuccessful completion of probation and/or parole

16. Measurement of other legislative outcomes • Annual report and brief mid-year report in ETO (no more quarterly reports) • Strategies to reduce duplication and fragmentation, involve families, etc • Costs and cost savings • Local performance measures • Yearly collaboration surveys to IOGs

17. Next steps – short-term April-May • Guidance documents provided for IOGs May-June • OMNI will build and train CMPs on ETO database • OMNI will work with CMPs who choose to modify local databases July • CMPs will enter FY10-11 annual report data into ETO • CMPs will begin tracking process indicators on ISST-referred families

18. Next steps – long-term FY11-12 • Develop plan to identify education and health/mental health outcome indicators • Identify steps to explore use of data in incentive formulas • Continue to streamline annual reporting

19. Questions? • What is needed for your discussions with IOGs about these changes?