Information for Effective Decision-Making: A Multi-Level Approach

1. Information for Effective Decision-Making:A Multi-Level Approach Nate Israel, PhD nathaniel.israel@sfdph.org

2. Context / Assumptions • The system is full of apparent tensions: cost vs quality, managed care vs individual control, competition for scarce resources • The system must respond to the needs of both client and political stakeholders • No single person or structure alone can adapt the system quickly enough to either political or client needs to both survive and markedly improve care

3. Surviving and Thriving • Distributing the responsibility for adaptation is necessary for client welfare and system sustainability • Typical top-down models of change often have short-term success at the cost of long term failure to adapt and short-term infighting • Reducing in-fighting and increasing the uptake of new behaviors requires new tools

4. Tools for Success • The Good News: We’ve spent the past 3 years developing these tools • Automated reports have been built (and are being replicated in NetSmart) to facilitate data-based discussions of what works and what can be improved (information forculture change) • As-needed reports are now available to address any clinical or functional issue we measure (information for stakeholder queries)

5. Feedback System: Goal • To allow persons at each level of the system to use the same data to identify and solve problems at their level of responsibility • To allow the system to identify the seriousness of specific issues and the magnitude of response needed to address the issue • To allow the system to identify successes to celebrate and replicate

6. CANS Feedback System: Multi-Level Information for Effective Decision-Making Client Needs and Strengths: Initial Needs / Strengths and Change over Time Clinician Needs and Strengths: Client level effectiveness, Cross-client effectiveness, Effectiveness vs Agency and System averages Supervisor Needs and Strengths: Client level effectiveness, Effectiveness of Clinical Supervisees, Relative Strengths by Domain of Strength and Need Program Needs and Strengths: Particular Needs of Clients at Entry; Change in Needs over Time (total change and change by item); Compliance and client Flow Characteristics (dis-engagement rate, time from entry to treatment, number of openings and closing) System Needs and Strengths: Program-by-program profile of Client Needs over Time; System Flow characteristics for individual Programs, Programs within a Level of Care, and the entire System; As-needed reports on any measured aspect of Clinical Symptoms or Functioning

7. Communication Structure Examples: • Client-Level Clinical Reports • Program-Level Reports • System-Level Reports Note: This overview is representative of the types of reports available, but is not an exhaustive description of the reports available.

8. Clinical Reports: Goal • To allow clinicians to quickly identify progress in goal achievement at the client and caseload level • To allow clinicians to quickly identify how their performance relates to that of other clinicians in the same agency

9. Clinical Reports 1) Change in Syndromes / Functioning over Time

10. Clinical Reports 2) Client and Caseload Level Change over Time

11. Program Level Reports: Goal • To allow supervisors to identify clinical successes and issues at the clinician and team level • To allow program directors to identify clinical successes and issues at each level of the program (clinician, supervisor, program) • To allow program directors to identify the unique needs of their client population and the agency’s effectiveness on meeting each need • To allow programs to learn from each other’s successes in effectively meeting clients’ needs

12. Program-Level Reports • RU-level change over time

13. Program-Level Reports • Agency Clinical Formulation

14. Program-Level Reports • Agency Clinical Formulation Over Time

15. Program-Level Reports • Agency Clinical Formulation: Summary Rating

16. System Level Reports • System Flow Reports • ACF / Clinical Effectiveness Reports • Need / Issue Based Reports

17. System Flow Reports: Goal • The most basic goal of a system flow reports is to identify the rate at which clients move through the system (Contract Performance Indicators) • Understanding flow at each level of care and program within a level of care allows you to identify system blockages • These analyses can also allow you to better understand disparities in access • Last, these analyses allow you to identify patterns of movement that may indicate effective/ineffective care

18. System Flow Reports: LOC

19. System Flow Reports • Typically look by RU • Can then calculate number of new clients, carryover clients, closed clients by quarter • Allows us to understand flow • Also allows us to monitor blockages in flow

20. ACF Reports: Goal • At the System level, these reports allow you to identify : • The effectiveness of the system in addressing client needs • The most prevalent conditions for treatment • Key partners needed to sustain change over time

21. ACF Reports: System Level

22. Need / Issue Based Reports • Many needs for information arise unpredictably • We can rapidly create brief reports on any clinical or functional issue we track • These reports can provide both data on the issue at hand, and contextual data on the persons affected by the issue, which may help us better understand the need

23. Need / Issue Based Reports • Key CANS data:

24. Need / Issue Based Reports Context: “Of those children and youth who have serious school attendance problems, 42% also had recent contact with the legal system because of delinquent behavior. Nearly twenty percent (17.8%) of serious truants were involved in criminal behavior (non-status offenses) during the past month. These children and youth may also have experienced neglectful or abusive child-rearing contexts. Over 20% of serious truants reportedly experienced parental neglect; 16% experienced parental physical abuse.”

25. Summary - Strengths • We currently have a system of reports useful to describe our children and youth on critical clinical, functional, and service dimensions • This system allows us to use similarly structured data at each level of the system to align system improvement with client needs • The system also allows us to respond in a structured way to unpredictable or sudden needs and demands