Using Outcomes in the Quality Improvement Process: Setting System Level Standards

1. Using Outcomes in the Quality Improvement Process: Setting System Level Standards Erik R. Stewart, Ph.D. Vice President, System Performance Eriks@hamilton.mh.state.oh.us Renee Kopache, M.S. Recovery Coordinator Reneek@hamilton.mh.state.oh.us

2. History/Purpose of Outcomes • Ohio Mental Health Outcomes Task Force (OTF) – 1996 • Charge • Develop a statewide approach to measuring consumer outcomes in Ohio’s publicly-supported mental health system Vital Signs (1998) • Outcomes Implementation Pilot Coordinating Group • Final recommendations were made in 1999.

3. Adult Outcomes Domains • Clinical Status • looks at symptoms that a person may experience from their illness and how much they interfere with their daily living. • Quality of Life • examines how “good” a person’s life is, and if their needs are being met. Also examines how much control a person has over the events in their life (empowerment).

4. Adult Outcomes Domains • Functional Status • This domain identifies how well a person is doing in the community including areas such as work, school and social relationships. • Safety & Health • Addresses how a person is doing physically and the amount of freedom they have from psychological harm from self and others.

5. Adult Outcomes: Instruments • Adult Consumer Form A • This form is used by consumers with a severe mental illness. (case management) • The survey consists of four parts: Quality of Life, Safety and Health, Symptom Distress and Empowerment.

6. Adult Outcomes: Instruments • Adult Provider Form A • consists of two parts: Functional Status and Safety & Health. • Provider’s observations and clinical judgments about the consumer’s social & role functioning, housing status, activities of daily living, criminal justice involvement, harmful behavior and victimization.

7. Adult Outcomes: Instruments • Adult Consumer Form B • Consists of three parts: Quality of Life (excluding empowerment, Safety & Health and Symptom Distress. • This shorter version of the Adult A form is used for adult outpatient clients. Providers of non-SMD consumers are not required to complete a provider survey.

8. Youth Outcomes Domains • Four primary areas or domains of assessment were selected for Ohio Scales: • Problem severity • Functioning • Hopefulness • Satisfaction with behavioral health services

9. Youth Outcomes: Instruments • Ohio Scales Three parallel forms: • Y-form is completed by the youth (self-report for ages 12 and older). • P-form is completed by the youth’s parent (or primary caretaker). • W-form is completed by the youth’s agency worker/case manager.

10. Administration Periods • Initial • 6 Month • 12 Month • Annually thereafter (Adult Consumer Form B: Initial and Discharge administrations only)

11. OCO History in Hamilton County • Data collection begins – September, 2001 • Regular and varied production reporting begins at same time • Production performance abysmal, though consistent with other participating areas of State

12. HCCMHB surveys agencies to discern impediments to production

13. Agencies React

14. HCCMHB representatives visit agencies to discern impediments to production

16. Impediments (in no particular order):1) Excuses: • POV boxes don’t turn on (batteries) • Staff refuse to complete or administer • Multi-million $ agency has only 1 printer • POV box times out too quickly

17. Impediments (in no particular order):2) Collection technology: • POV data system interface with other agency data system(s)

18. Impediments (in no particular order):3) Lack of agency/staff “buy-in” • We know we do good work • We don’t care about consumer outcomes

19. Impediments (in no particular order):4) Lack of products/information to staff following administrations (initially) • Staff frustration with initial software • Introduction of ARROW with use limited to POV • Immediate red flags printout • Weekly reports to staff • Consistent supervisor response to problems • Use of data in “assessment update printout”

20. Impediments (in no particular order):5) Diffusion of responsibility(a social phenomenon that occurs in groups of people when responsibility is not explicitly assigned) • Request made by QA/Outcomes member to resurrect regular IT/IS staff meetings at HCCMHB. Solicitation made. ONE response. • Lack of departmental ownership

21. Impediments (in no particular order):6) Agency-specific challenges • Central Point of Access • New administration at agency intake? • Paper and pencil administration • Program support staff enter data

22. QA Approach Introduced • Quarterly indicator measuring Initial Adult Provider compliance added to existing indicators

23. ODMH Introduces Missing Data Reports - Summer, 2003 • Hamilton County Performance • 23% Adult OR/OE • 34% Youth OR/OE

24. October, 2003 - HCCMHB and Agencies Form PI Sub-committee to Develop Production Logic

25. February, 2004 – Ohio Department of Mental Health Presents State Plan for Upcoming PI Requirements

26. Plan Act Do Check PDCA Cycle

27. April, 2004 – Proposed Logic for OCO Production Measures Shared With Membership

28. Initial Adult Consumer Production Analysis

29. Initial Adult Consumer Production Analysis • Claims data compiled for quarter under review

30. Initial Adult Consumer Production Analysis • “Pre-quarter period” is determined by subtracting 120 days from first date of service in period

31. Initial Adult Consumer Production Analysis • Review of claims during pre-quarter period is conducted. Those exhibiting claims are removed

32. Initial Adult Consumer Production Analysis • Unduplicate list by selecting case with highest order service

33. Initial Adult Consumer Production Analysis • Create date range for acceptable OCO administrations

34. Initial Adult Consumer Production Analysis • Merge Adult Consumer A and Adult Consumer B OCO administrations • Match by date when acceptable date exists • Determine production ratio/percentage

35. Period under review (4th qtr. FY 2005)

36. 1st service in period May 11, 2005

37. Absence of service determined for 120 day period prior to first service (ensures client is new or in new episode of care)

38. Outcome administration expected/sought between March 12 and June 24

39. Initial Measures Adult Consumer HCCMHB OCO Production Performance Improvement Measures

40. Initial Measures Adult Consumer Adult Provider Youth Consumer Youth Provider Youth Parent Six-Month Measures Adult Consumer Adult Provider Youth Consumer Youth Provider Youth Parent 12-Month Measures Adult Consumer Adult Provider Youth Consumer Youth Provider Youth Parent Long-Term Measures Adult Consumer Adult Provider Youth Consumer Youth Provider Youth Parent HCCMHB OCO Production Performance Improvement Measures

41. June, 2004 – Contract Language Established Obligating Agencies to 70% Compliance on OCO Administrations

42. July, 2004 – HCCMHB Issues First “Test” Measures

44. PI Products Provided to Agencies:

47. Spreadsheet containing: • UCI • First service in period date • Last service in period date • OCO time period parameters • Submission status • Date of administration for submission • Age of client • Service type by numeric rank • Instrument type of submission • Admission date

48. Agencies may formally dispute findings on a case-specific basis

49. Measures allow agencies to look closely at specific cases missing administrations to discern issues

50. Had to educate agencies on intricacies of logic Tracking based upon most recent intake date