Anita Keyes , Minnesota Department of Human Services

1. Using Citizens Review Panels in the Child andFamily Services Review (CFSR) and Program Improvement Plan (PIP) Process Anita Keyes, Minnesota Department of Human Services Phyllis Fulton, N. C. Division of Social Services Ernestine Moore, Michigan Citizens Review Panels

2. Ongoing Quality Assurance of the Child Protection System Case Reviews Using the Well-Being Outcomes: a Role for the Citizen Review Panel Minnesota Citizen Review Panels Minnesota Department of Human Services 444 Lafayette Road North St. Paul, Minnesota 55155-3832 May 27, 2004

3. The Design of the FederalChild and Family Services Review Three Domains Child Safety Permanency for Children Child and Family Well-Being Child Safety Outcomes Children are, first and foremost, protected from abuse and neglect Children are maintained in their homes whenever possible and appropriate Permanency Outcomes Children have permanency and stability in their living situations The continuity of family relationships and connections is preserved for children Child and Family Well-Being Outcomes Families have enhanced capacity to provide for their children’s needs Children receive appropriate services to meet their educational needs Children receive adequate services to meet their physical and mental health needs

4. The Three Well-Being Outcomes of the Federal Child and Family Services Reviews Well-Being Outcome One Families have an enhanced capacity to provide for their children’s needs. Well-Being Outcome Two Children receive appropriate services to meet their educational needs. Well-Being Outcome Three Children receive adequate services to meet their physical and mental health needs.

5. The well-being outcomes are a good fit for the Citizen Review Panel because: They involve gathering information outside of the county and state child protection agencies and systems, for example from schools, medical care providers or therapists. These outcomes are community oriented. All states that have been through the federal Child and Family Services Review have received ratings of “area needing improvement” in many of the well-being outcomes.

6. Safety and Permanency Outcomes These outcomes more often relate to compliance with statutes, legal time limits and social work best practice. Safety and permanency outcomes are more suited to be reviewed by county agency directors, managers, supervisors, workers and state external reviewers on an ongoing basis. The results of their reviews can be monitored by the Citizen Review Panels.

7. Ongoing Quality Assurance What do you need to conduct reviews ? Citizen Review Panel and Child Protection Services’ Resources: • A number of well-trained Citizen Review Panel reviewers able and willing to commit time • Format and instructions modeled after the federal Child and Family Services Review (CFSR) instrument and instructions • Support, cooperation and participation from county and state child protection staff • Coordination with county and state staff to obtain information and data • Access to child protection case records • Access to other relevant records

8. What do you need?(continued) Community Partners/Resources: • Recipients of Services • Mental health care providers • Domestic violence victim’s advocates • Medical care providers • Schools • Faith-based organizations • Law enforcement • Officers of the Court (judges, attorneys, guardians ad litem, foster parents, corrections officers)

9. Ongoing Quality Assurance How will you conduct the reviews? • Citizen Review Panel Activities • Identify needs and assess services: • Attend case review orientation and training sessions • Read case files, reports and records • Interview key people involved in the case • Use format modeled after instrument used in CFSR well-being outcomes section • Questionnaire/Survey • Focus Groups

10. How will you do it?(continued) Report findings, make recommendations and monitor: • Process findings with entire panel • Provide information and make recommendations in writing to county social service agency • monitor the county agency’s response to the recommendations • Provide written follow-up reports to the social service agency as necessary • Include information in Citizen Review Panel’s annual report

11. Ongoing Quality AssuranceWhat are you looking for during the reviews? From the child protection system: • Needs and services of the child, parents and family • Child and family involvement in case planning • Worker visits with the child • Educational needs of the child • Physical health needs of the child • Mental health needs of the child From the Citizen Review Panel: • Identify unmet needs, identify what is working well and make recommendations to improve outcomes

12. OutcomesWhy are you doing the reviews? Outcome Well-Being One: Families have an enhanced capacity to provide for their children’s needs. Outcome Well-Being Two: Children receive adequate services to meet their educational needs. Outcome Well-Being Three: Children receive adequate services to meet their physical and mental health needs.

13. Why are you doing the reviews?(continued) IF citizen review panel members make recommendations to the county social service agency based on comprehensive case reviews that focus on the well-being outcomes, and monitor the county agency’s responses to the recommendations. THEN children’s educational, physical and mental health needs will be consistently identified and met, and families will have an enhanced capacity to provide for their children’s needs.

14. Chisago County Citizen Review Panel Ongoing Quality Assurance Pilot Project Focus was on well-being outcome two: Children receive appropriate services to meet their educational needs.

15. Questionnaire Developed Based on the Federal Child and Family Services Review Instrument: • WB Outcome 2: Children receive appropriate services to meet their educational needs • Item 21: Educational needs of the child • Core Questions (mandated), A - B7 • Additional Core Questions - Best Practice, C - D6 • Twenty-three Exploratory Issues articulated Key words defined: • Absent • Tardy • Truant

16. Questionnaire(continued) Rating for Item 21: Educational needs of the child: • Strength • Area Needing Improvement • Not Applicable Rating for outcome WB 2: Children receive appropriate services to meet their educational needs: • Substantially Achieved • Partially Achieved • Not Achieved • Not applicable

17. Criteria for cases selected • Children in out of home placement • Truancy cases (minority of cases from this category) • Children ages 5 to 18 • Hand-select cases to avoid having workers, foster parents or schools over-represented • Currently open or closed • One child per family/case • The period under review is September 1, 2002 to May 31, 2003, the 2002-2003 school year. Cases should be open at some time during that time period

18. Duties of Participants • County on-site coordinator • Panel members • Case workers • State coordinator

19. Review Teams • Five review teams were comprised of two panel members each. • Nine cases were reviewed.

20. Orientation Citizen Review Panel and Workers • Session One - Two Hours • Introductions • Background/History • Qualitative Review Instrument/Questionnaire and Instructions • Assign teams • Session Two - One and One Half Hour • Interviewing Skills • Define role of Citizen Review Panel reviewer, purpose of review and articulate what will be done with the information gathered • Assign cases

21. Timeline • July 2003 • Planning meeting • August & September 2003 Citizen Review Panel Meetings • Orientation and training • September 2003 through April 2004 • Conduct case reviews & discuss process/progress at monthly meetings • February & March 2004 • Debrief the cases at the Panel meetings • April 2004 • Evaluate review process at Panel meeting • Write final report • May 2004 • Present results to Chisago County Health and Human Services • Present process and results at the National Citizen Review Panel Conference, Lexington, Kentucky

22. Results of the Case ReviewsBased on the results of the case file reviews and interviews, the nine cases were rated: Item: Educational needs of the child Strength 6 Area Needing Improvement 3 Outcome WB Two: Children receive appropriate services to meet their educational needs Substantially Achieved 6 Partially Achieved 3 Not Achieved 0

23. Evaluation by the Citizen Review Panel and County Social Services’ Staff • Overall impressions of process and experience • Preparation for case review process • The actual case review process • Case review form (instrument) • Support from county and state staff

24. Other Applications • Areas identified as partially achieved or not achieved and in the Program Improvement Plan • Social work practice areas that need to be implemented or improved • Evaluation of what is already in place before implementing initiatives or new mandates • Review of problem areas particular to a county agency or area (truancy, foster care re-entry…) • Evaluation of issues to support the request for funds, services or employees • Preparation for a federal or state child and families services review

25. Other Applications(continued) • Define the issue to be reviewed • Determine which federal Child and Family Services Review (CFSR) outcomes and items are relevant • Adapt the CFSR instrument to meet the needs of the issue reviewed. Include the instrument’s exploratory issues and add your own • Adapt the review processes suggested in the CFSR manual and used by the Chisago County Citizen Review Panel

26. Assessing Services(Example) • Reviewers: _________________________ Child: DOB___________ gender_______ • Period under review: ______________ Date of review: ______________________ • We want to determine how child protection services are addressing the mental health, physical health and child developmental needs of children, ages zero to three years old, who are in out-of-home placement. What services are being provided? • Assessment of Needs/Provision of Services Chart • Identified Needs Services Provided Unidentified Needs Services Needed/Not Provided • Exploratory Issues • What type of assessment process was used to identify needs (for example, a psychological evaluation and/or discussions with relevant parties)? • How adequate was the assessment in covering all relevant areas and in identifying needs? • What are/were the underlying needs associated with more obvious needs or presenting problems? • What services have been/are being provided in relation to current needs? • How appropriate are/were the services provided in relation to the identified needs? • How accessible and available are/were services (for example, location, schedule, cost)? • To what degree are/were the services provided meeting the identified needs? • How accessible to foster parents is/was the case worker? • How appropriate is/was the child’s placement setting (for example, family-like and suited to the child’s needs)? • Rating Services Provided by Child Protection Services: • Reviewers should rate this item for the period under review, although in responding to questions, reviewers should consider initial assessments of needs that were conducted outside the period under review, and ongoing assessments during the period under review. • Reviewers should note in particular whether the following services are/were needed and provided: (1) community-based family support services, (2) family preservation services, (3) time-limited family reunification services, and (4) adoption promotion and support services. • Reviewers should take special care to note whether the services provided are/were appropriately matched to the identified needs. • Reviewers should explore the accessibility and availability of services being provided (for example, location and schedule). • Assessment of needs may take different forms (for example, a psychological or social evaluation conducted by another agency or by contract purchase). Reviewers also may find evidence during interviews with caseworkers or service providers that identifiable efforts were made to assess needs through the case planning process (using a process other than formal assessment) and the caseworker has an in depth understanding of the needs of the child and family upon which to base the case plan. • Rating (check one) • Strength • Area Needing Improvement • Provide documentation

27. Exploratory Issues • What type of assessment process was used to identify needs (for example, a psychological evaluation and/or discussions with relevant parties)? • How adequate was the assessment in covering all relevant areas and in identifying needs? • What are/were the underlying needs associated with more obvious needs or presenting problems? • What services have been/are being provided in relation to current needs? • How appropriate are/were the services provided in relation to the identified needs? • How accessible and available are/were services (for example, location, schedule, cost)? • To what degree are/were the services provided meeting the identified needs? • How accessible to foster parents is/was the case worker? • How appropriate is/was the child’s placement setting (for example, family-like and suited to the child’s needs)?

28. Rating Services Provided • Reviewers should rate this item for the period under review, although in responding to questions, reviewers should consider initial assessments of needs that were conducted outside the period under review, and ongoing assessments during the period under review. • Reviewers should note in particular whether the following services are/were needed and provided: (1) community-based family support services, (2) family preservation services, (3) time-limited family reunification services, and (4) adoption promotion and support services. • Reviewers should take special care to note whether the services provided are/were appropriately matched to the identified needs. • Reviewers should explore the accessibility and availability of services being provided (for example, location and schedule). • Assessment of needs may take different forms (for example, a psychological or social evaluation conducted by another agency or by contract purchase). Reviewers also may find evidence during interviews with caseworkers or service providers that identifiable efforts were made to assess needs through the case planning process (using a process other than formal assessment) and the caseworker has an in depth understanding of the needs of the child and family upon which to base the case plan. • Rating (check one) Strength Area Needing Improvement • Provide documentation

29. Win, Win, Win!Panel Members: • Appreciate the complexity of the work of the social workers • Learn the needs of children and families in their communities • Provide meaningful, challenging work and recommendations • Bring community standards to the child protection system • Advocate for needed services • Act as knowledgeable ambassadors to protect children in their communities

30. Win, Win, Win!Child Protection Agency: • Workers learn what they are doing well and what they could be doing differently • It’s good “practice” for state and federal reviews • Helps keep agency accountable to their clients and their community • Promotes communication with citizens and community • Improves social work practice • Assists the agency to focus on reaching the goals of its Program Improvement Plan • Citizens may identify issues those in the child protection system could miss

31. Win, Win, Win! Children’s educational, physical and mental health needs will be consistently identified and met, and families will have an enhanced capacity to provide for their children’s needs.

32. CCPT and Federal Child Welfare Reviews Child Advocates Joining for The Children

33. Using CCPT in Federal Reviews 1. Include CCPT members on stakeholder committee 2. Include community assessments from CCPTs in state self assessment 3. Integrate information contained in CCPT End of Year Reports (EYR) in state’s self assessment 4. Interview CCPT members as a part of federal review process 5. Disseminate final reports to CCPT

34. Benefits of Including CCPT Members on Stakeholder Committees • Review of cases give CCPT members first hand knowledge of factors that contributes to the risk of children. • Adds a community perspective to the committee • Teams member may be more objective about the overall state of child wellbeing

35. Community Assessment • In preparing for the review states should request community assessments from all CCPTs. While the review sites are limited in numbers, a community assessment from all CCPTs will provide data that reflects the state of child protection statewide.

36. Why Interview CCPT Members • Enhances the federal review process by adding a community response • Integrates agency, family and community

37. Integrate End of Year Reports in Program Improvement Plan (PIP) • EYR indicate state trends. • EYR identifies barriers that impact child wellbeing. • Since EYR are completed annually the EYR can be used to track progress in eradicating barriers by comparing year by year.

38. Sharing Review Outcomes Sharing the final review report with the CCPT will provide CCPT members with information about the areas of child protection that need to be address from a federal perspective.

39. Michigan’s Citizen Review Panels In The CFSR and PIP Process

40. Michigan’s CRP Structure • Three Panels • CRP on Prevention • CRP on Protective Services, Foster Care and Adoption • CRP on Child Deaths

41. CRP Steering Committee • Purpose: coordinate and guide work of the 3 panels • Composition: chairs of individual panels and staff to the panels

42. CFSR Dates • On-site review: September 9 – 13, 2002 • Review process began: March 2000

43. Statewide Assessment Review Team • Review data • Provide objective in-put and analysis • Question without recourse

44. Selection of Sites for Review • Ensure accurate representation of State’s child welfare service delivery system for State selected sites

45. On-site Review • State Team Member • Review documents • Interview stakeholders • Complete Federal Review forms • Assist Federal Team members with understanding Michigan’s policies and practices • Assist with Site Summary Reports

46. Review Final Report • Accuracy • Findings not identified during Statewide Assessment • Suggestions for improvement

47. Review PIP • Actions proposed • Probability of success • Other options

48. CRP Initiatives • Parent Handbook • Child/youth Handbooks • Screening, assessment and treatment of children in foster care • Tracking children with mental health needs who are experiencing multiple moves • Tracking data on permanency for children whose parental rights have been terminated

49. CRP Initiatives (continued) • Tracking inter and intra –agency actions to ensure availability of services needed to reunite families: housing, substance abuse treatment, mental health treatment, and support services • Advocating funding “normalized” supports for children: music lessons, athletic opportunities, girls/boys club memberships

50. CRP Initiatives (Continued) • Community engagement in prevention and protection • Case review of child abuse and neglect cases where deaths occurred before, during, and after child welfare system involvement (prevention services, protective services, foster care services, adoption services)