Connecticut State Department of Education Bureau of Special Education

1. Connecticut State Department of Education Bureau of Special Education Annual Back to School Meeting September 17, 2014 Breakout Session: Supporting Student Success Through Positive School Climate (Actionable, Student Level, Social-Emotional Data) Alvin F. Larson, Ph.D. Meriden Public Schools Meriden, Connecticut Please complete your Confidentiality Sharing Exercise

2. CSDE’s Student Success Plan Core Components • Academic: “attitudes needed to be an effective learner in school”; • Career: “student will investigate their own interests … • and/or career goals”; • and today’s primary topic: • 3. Social-Emotional: “maintaining positive interpersonal relationships, managing feelings and emotions”. • Today’s Purpose • Demonstrate two tools that provide actionable student level data that initiate and guide differentiated social-emotional intervention.

3. Taking Action with Social-Emotional Data (Student Success Plan) Current Education Data Use • Pro-Social outputs but • some behavior problems • Absenteeism • Bullying • Dropouts • Crime • Suicide Tools • Pro-Social outputs but • less behavior problems • Absenteeism • Bullying • Dropouts • Crime • Suicide If we add comprehensive Social-Emotional Support tools such as #1 Confidential School Climate Survey MPS Cares #2 Confidential Getting to Know You survey Social-Emotional, Perseverance, Mindset and Motivation Actionable Data for Monitoring and Intervention

4. #1 Meriden School Climate Survey for Students (MSCS-SV) based on National School Climate Standards and designed for school psychologists • Purpose (more pro-social outputs and less behavior problems) • A timely response system to identify students who perceive they are experiencing a social-emotional problem; • New insights of any student's perceptions, especially those students receiving counseling services (Special Education and Non-Special Education); • Estimate the proportion of latent, social-emotional at-risk students that remain unidentified (~5% of your total student population); and • Provide measures of "school climate" to meet state legislative requirements • MSCS-SV was approved by CSDE.

5. Current Thinking on Bullying and Other Mean Behaviors • Whole school interventions recommend victims tell an adult, students rarely do1 • Relational aggression is a bullying problem for adolescent girls. Social exclusion and friendship manipulation is associated with depression and suicidal ideation 2 • To avoid many academic, social and emotional problems, we must become more adept at identifying possible victims and bullies 3 • Developmentally sensitive strategies for avoiding and responding to bullying-type situations are needed as well as seeking input from adolescents … bullying is a relationship problem requiring improved relationship and coping skills. 4 • Bauman, S., (2010). Cyberbullying in a rural intermediate school: An exploratory study. The Journal of Early Adolescence, 30(6): 803-833. • Raskauskas, J., & Stoltz, A.D., (2004). Identifying and intervening in relational aggression. The Journal of School Nursing, Aug;20(4): 209-215. • Lyznicki, J.M., McCaffree, M.A., & Robinowitz, CB., (2004). Childhood bullying: implications for physicians. American Family Physician, Nov 1;70(9): 1723-1728. • Bostic, J.Q., & Burnt, C.C., (2011). Cornered: An approach to school bullying and cyber bullying, and forensic implications. Child and Adolescent Psychiatric Clinics of North America, 20 (3), 447-465.

6. MSCS-SV: A Developmentally Sensitive Method (customized software and database) • The online MSCS-SV is considered a needed complement to whole school character programs (such as PBIS) • Confidential, we know who logs on (student identification number); and • before taking the survey, students are informed … • “All of your answers are confidential. That means we cannot share your answers with anyone, unless you tell us about danger or someone is going to be hurt. We must keep all students safe.” • Now, look at your Confidentiality Sharing Exercise • How many told other people about your confidential conversations? • Scenario #1 (General Confidentiality) • Scenario #2 (Possible Bullying) • Scenario #3 (No Friends) • Scenario #4 (No Home Support) • This is also my problem; currently, teachers do not have access to MSCS-SV data (only counselors). What data can be ethically shared?

7. Students in Perceived Social-Emotional Crisis • Trigger emails produce actionable data

8. Confidential and Responsive Online School Climate Survey The following student quotes are samples of student “trigger emails”. They deal with suicide, harm, threats, relational and racial issues. The underlined segments, added by the presenter, align with national qualitative research findings.

9. Responsive Online School Climate Survey for Students

10. Matched Scores on Factor 2 “Safety” (hits/threats or mean rumors) October to June - Of the 1,959 students who took both climate surveys: 1,842 New student application: “MPS Cares” open all year … 1,842 1,727 117 117 115 88 29

12. Meriden Cares … Sample Email text to Staff Staff email alert from … Noreply @ … Meriden BOE MPS Cares The following student submitted an Meriden Public Schools Cares Entry Student Student ID: xxxxx Name: STUDENT NAME School: SCHOOL Question Responses: Problem with friends Yes People are mean Yes Other problem No Comments: (student comment is optional) Trigger emails are automatically sent if any “yes” is checked. This site is only available to students and is only accessible on Meriden Public Schools computers. (Student View)

13. At the request of administrators and counselors, we added an additional trigger email alert related to “no friends” • When a student responds with a “1” or “2” on this item, a trigger email is immediately sent to: • School Psychologists • School Social Workers • School Administrators • Office of Research and Evaluation

14. Scientific Validation of MSCS-SV National School Climate Standards recommend we build instruments that are valid, reliable and measure latent psychological constructs such as: engagement in school, respect, caring, fairness and pro-social behavior. Each latent factor (attitudes, beliefs and values) influences how each student responds to a sub-set of questions. The Factor loadings are correlations with each attitude/belief/value. Item 37 .719 .711 Factor 1 Caring Item 30 .694 Item 36 .767 Item 19 .706 Item 14 .563 Item 9 Factor 4 Friends

15. Scientific Validation of MSCS-SV Validity and Reliability Standards

16. Scientific Validation of MSCS-SV • Exploratory Factor Analysis • LEA Level (2011, 2012) • Independent Confirmation Factor Analysis • University Level (2013) • comparative fit index (CFI) ≥ .90 for Seven Factor Model • Teachers Motivate & Care (r = .914) • Safety (r = .758) • Respect Differences (r = .759) • Parent Values School (r = .746) • Parental Support (r = .721) • Caring Friends (r = .749) • Aggression (r = .720)

17. FIGURE 1  Cross-Sectional Non-Matched Average Factor 1 Scores: Teachers Motivate and Care by Grade Level Most positive perceived climate (Factor-based scores: 1-5) Lowest perceived climate Teachers Motivate and Care, is composed of 11 items (r = .914) Students perceive teachers as helpful, fair, caring and motivating; teachers listen to them, tell them when they do a good job and students are happy to be in this school.

18. FIGURE 2 Cross-Sectional Non-Matched Average Factor 3 Scores: Respect Differences by Grade Level Most positive perceived climate (Factor-based scores: 1-5) Lowest perceived climate Factor 3: Respect Differences is composed of 5 items (r = .759) Students respect each other's differences, they see skin color as a potential problem; other problems are fighting and students being mean to others.

19. FIGURE 3  Cross-Sectional Non-Matched Average Factor 5 Scores: Perceived Parental Support by Grade Level Most positive perceived climate (Factor-based scores: 1-5) Lowest perceived climate Factor 5: Parental Home Support is composed of 4 items (r = .721) Student perceives s/he has a place and time to do his/her homework and there is a parent who listens and will help with homework if needed.

20. FIGURE 4 Cross-Sectional Non-Matched Average Scores of Stable Factors across grade levels: Aggression, Parent’s Value School, Friendship and Safety by Grade Level Most positive perceived climate (Factor-based scores: 1-5) Aggression Towards Others Parents Value School Caring Friends Safety Lowest perceived climate Factor 7: Aggression Towards Others is composed of 3 items (r = .720); in school, an unacceptable coping skill. Student claims to have hit, pushed or spread mean rumors about others during the past few months, sometimes in anger. Factor 4: Parent Values School is composed of 4 items (r = .746) Students perceives s/he has a parent or other adult at home who cares about his/her school work, and wants the student to do his/her best and follow school rules; the student also feels s/he will be successful in life. Factor 6: Caring Friends is composed of 4 items (r = .749) Students perceive they have a friend their own age who cares about them and they trust and can talk about their problems. Factor 2: Safety is composed of 7 items (r = .758) Students perceive other students as hurting their feelings, saying mean things, possibly hitting and threatening; they worry about their safety.

21. TABLE 1 Average Survey Factor differences for Special Education versus non-Special Education Students by Climate Factor and Grade Level (significant > .2)

22. AERA 2014 Annual Meeting FIGURE 5 Matched Factor-Based Scores for Teachers Care and Motivate from September 2011 through September 2013: Eight Grade-Level Cohorts • Seasonal Variations • During the school year (solid lines) all decrease except one grade 4 cohort and one grade 10 cohort. • Summer (dotted lines) all increase; optimism that this school year will be better. • Matched factor-based scores could be used to evaluate interventions. Gr 4 to 6 Gr 3 to 5 Gr 5 to 7 Gr 6 to 8 Gr 8 to 10 Gr 10 to 12 Gr 7 to 9 Gr 9 to 11

23. AERA 2014 Annual Meeting • Most matched cohorts show increasing matched scores • Decreasing (cohort grade 8 to 10); • About the same (cohorts: grade 5 to 7; grade 7 to 9; and grade 9 to 11); • Increasing (the other four cohorts, despite non-matched decreases). • Matched factor-based scores could be used to evaluate interventions. FIGURE 6 Matched Factor-Based Scores for Respect Differences from September 2011 through September 2013: Eight Grade-Level Cohorts Gr 4 to 6 Gr 3 to 5 Gr 5 to 7 Gr 8 to 10 Gr 6 to 8 Gr 7 to 9 Gr 9 to 11 Gr 10 to 12

24. Individual Student Profiles AERA 2014 Annual Meeting Safety • MSCS-SV historical data • Identify changes in student perceptions • Identify unknown students who are latently “at-risk”, and • Provide additional student perceptions of each counselor’s current caseload

25. School Psychologists Log in Screen for MSCS-SV Log in to the School Climate Survey Website: Email: Your Meriden email address Password: Supplied

26. School Psychologists Ethic Statement for MSCS-SV Staff will need to agree to the Protocol for Sharing Confidential Information

27. School Psychologists View of MSCS-SV Database Staff can search results by student name, survey administration or by Factor

28. School Psychologists View of MSCS-SV Database Select student name to view Historical Profile

29. School Psychologists View of MSCS-SV Database Historical Profile for student in grade 9

30. School Psychologists View for MSCS-SV Database Staff can select Trends to view one year changes

31. School Psychologist View of MSCS-SV Database Staff selects NOTIFICATION Staff highlights student name if accepting responsibility for follow-up

32. MPS Cares Notification Accept Notification

33. Percent of Social-Emotional At-Risk Students • 7 % perceived hit/threat or mean rumors in fall (6% matched scores) • 10% perceived hit/threat or mean rumors in spring (6% matched scores) • 4.8 % unknown, latent, students self-report social-emotional at-risk • 9.8% with “low” scores (-2 σ below grade level mean) • ______ • 15.1% unduplicated count of all students base on • at-risk by trigger email, • “low score” (-2 σ) or • “substantial change” in factor-based scores • A working group of school psychologists and social workers have developed protocols to respond to student trigger emails and historical database reports.

34. To: Administrators, School Psychologists, Counselors and Social Workers • Date: June 2014 • Re: Protocol for Trigger Emails for Meriden Student Climate Survey • The following expectations serve as our protocol for responding to the trigger emails in Meriden Public Schools. • Protocol for Trigger Emails • Students that are the subject of the trigger email will be met with within two school days. • The attached Response Interview Forms will be used. • Trigger email responses will be the collective responsibility of all school psychologists, counselors, and social workers in the building. • Student Support Staff shall share the active Response Interview Forms with colleagues in other Meriden schools if a child transfers during the school year. • After two years of inactive files, the Response Forms and emails should be destroyed. • The Meriden Student Climate Survey shall be administered in October and March in order to ensure adequate time for staff to support the student.

35. Meriden Student Climate Survey – Response Interview Form On the survey, you answered a question that met the criteria to look further into. The question was (read from survey) Can you give me an example of what that looks/ sounds like? Who has said/ done this? When has it happened? How often? Was anyone else there? Does it continue to happen? Any other information you would like to share? Overall do you feel safe at school? Yes or No ____________________________________________________________________________ Response/ Solution by interviewer Check what is applicable: ____Parent contacted: Date___________ ____ Periodic check-in ____Administrator notified ____ Follow up counseling ____Teacher notified ____ Follow up with perpetrator ____No Follow up required at this time ____Other

36. AERA 2014 Annual Meeting Case 1 there are some people who call me names and … people start rumors about me that idont like sometimes i want to die and kill myself before the next day. i feel like everyone hates me and for no reason. ive sometimes been nice but when they bring up the rumor i say realy bad things to keep them away … one day i was thrown a note saying im ###. and it was my by bestfriend ... i lost the only thing that i can trust. i was alone for a long time. if this is unhealthy then pleace help me … Student was in counseling within 30 minutes of receiving this trigger email _______________________________________________________________________ How many student suicides … ? How many violent student assaults … ? Ex post facto analysis: Were the “signs” there? How could we have known/prevented? Just ask students via confidential survey. LEAs can be aware/preventative … “seek ‘developmentally appropriate’ input from adolescents”. Significance of MSCS-SV … a chance to be pro-active and preventative

37. AERA 2014 Annual Meeting Significance of MSCS-SV • Identifying latent at-risk students (for counseling services; 4.8%) • Identifying aggressive students (potential bullies also need counseling; 2% to 3%) • Ability to follow students over time (matched scores) • School climate has seasonal variations • Identifying disengaging students (dropouts) – being proactive • Historical data/profiles “very helpful” with current caseload • Behavioral screen of future problem behaviors • Protective and risk factors of school climate and bullying victimization • (Gage, et. al. 2014; in press, School Psychology Quarterly) • Other possibly criminal or violent events (including community/family) • Summary • Established, valid, reliable and actionable tool/system • Less behavior problems with improved social-emotional health

38. #2 “Getting to Know You” Survey (customized software and database) Second Year Research Project Purpose: • Help foster teacher-student relationships (teacher care); • Monitor student “interests … and/or career goals”; and • Provide guidance on how to modify instructional methods to maximize and maintain student perseverance/motivation over time; • motivation is “needed to be an effective learner in school”. Attributes: • Teacher-student relationships and student perseverance/motivation are positively correlated with student cognitive achievement. • About 12 to 15 minutes, once a year (September or as new students enter). • Teachers/staff only have access to their own student’s data. 38

39. Getting to Know You survey (guided by research) Research: to increase motivation, provide students with a … • Perceived caring environment • Students want teachers to know about them personally • Non-evaluative, constructive but positive feedback • Student can do better … with effort (mindset) • Safe ways to take risks and okay to make mistakes • Balance the task, challenging but doable • Make the topic/task something they like • Useful,important and find role models

40. Getting to Know You survey Ask students about their interests, beliefs and values: Activities: Do you play a musical instrument ... sport … sing … other activity? What do you plan to do after high school? How much do you like … math … reading … art … music? Motivation: What is important? … math … reading … art … music? How hard did you work last year? (effort) … math … reading … art … music? Perseverance:… mistakes are OK, as long as you’re learning … if a person works harder, he/she can do better … when class work gets hard, I just work harder Mindset: My success is related to my effort, or people can’t change.

41. Getting to Know You survey (Pilot Data: Factors and Scales) Perseverance Mindset Motivation (like + effort + perceived importance) - by subject area • Math • ELA Reading and Writing • Science • Social Studies • Art • Music • Physical Education • Technology

42. FIGURE 8 Cross-Sectional Non-Matched Average Factor-Based Scores: Perseverance Most positive self concept Lowest perceived self concept Perseverance (Pilot Data 2013-2014)

43. FIGURE 9 Cross-Sectional Non-Matched Average Factor-Based Scores: Mindset Most positive self concept Lowest perceived self concept Mindset (Pilot Data 2013-2014)

44. FIGURE 10 Cross-Sectional Non-Matched Average Motivation Scores: Mathematics Most positive self concept Lowest perceived self concept Motivation in Math (Pilot Data 2013-2014)

45. FIGURE 11 Cross-Sectional Non-Matched Average Motivation Scores: English Language Arts (ELA) Most positive self concept Lowest perceived self concept Motivation in ELA (Pilot Data 2013-2014)

46. FIGURE 12 Cross-Sectional Non-Matched Average Motivation Scores: Science Most positive self concept Lowest perceived self concept Motivation in Science (Pilot Data 2013-2014)

47. FIGURE 13 Cross-Sectional Non-Matched Average Motivation Scores: Art Most positive self concept Lowest perceived self concept Motivation in Art (Pilot Data 2013-2014)

48. FIGURE 14 Cross-Sectional Non-Matched Average Motivation Scores: Music Most positive self concept Lowest perceived self concept Motivation in Music (Pilot Data 2013-2014)

49. FIGURE 15 Cross-Sectional Non-Matched Average Motivation Scores: Physical Education Most positive self concept Lowest perceived self concept Motivation in Physical Education (Pilot Data 2013-2014)

50. TABLE 2 Average Survey Factor differences for Special Education versus non-Special Education Students by “Getting to Know You” Scale and Grade Level (significant > .2)