Individualizing Intervention Coaching to Increase Fidelity of Implementation

1. Individualizing Intervention Coaching to Increase Fidelity of Implementation Joseph W. Pieri, MS, University of Pittsburgh Shannon B. Wanless, PhD, University of Pittsburgh Danielle Marks, BA, University of Pittsburgh Sara E. Rimm-Kaufman, PhD, University of Virginia Coaching Teachers in Evidence-Based Classroom Practices: New Approaches to Increase Engagement and Impact Moderator: Karen L. Bierman May 28, 2013 Society for Prevention Research Conference San Francisco, California This study was supported by the the Responsive Classroom Efficacy Study (RCES)) School of Education

2. Individualizing Intervention Coaching • Social Emotional Learning (SEL) programs can lead to significant improvements in social-emotional skills, attitudes, behavior and academic performance when implemented with high fidelity or high quality (Durlak, Weissberg, Dymnicki, Taylor and Schellinger, 2011) • Eighty percent of schools are not implementing SEL programs with high levels of fidelity (Durlak et al., 2011) • The quality or fidelity of implementation (FOI) can be improved through the use of program coaches (Wehby, Maggin, Partin, and Robertson, 2012 ; Kretlow and Bartholomew, 2010; Kam et al., 2003)

3. What is a coach? Coaches work with teachers providing: Program training Feedback, assistance with issues related to classroom management On-going support through troubleshooting issues regarding implementation (Wehby et al., 2011) Empirical data reveals effective coaching is often marked by: Multiple training sessions High engagement between coaches and teachers On-going classroom observations Specific feedback (Wehby et al., 2011; Kretlow and Bartholomew, 2010)

4. Why is this important? Typically, coaches often rely on experience when responding to presenting psychological needs of school personnel Teachers are more responsive to a coach’s instruction when they view their relationship with the coach as being positive (Wehby, et al., 2011) This exploratory study can help unpack the coaching process by revealing how coaches interact with school staff in their training and may help the field move towards more evidenced based practices

5. Justifying Themes Research reveals fidelity of implementation rates can be enhanced or compromised by a host of contextual influences: Principal support School climate Teacher buy in Quality of teacher training (Domitrovich, Bradshaw, Poduska, Hoagwood, Buckley, Olin, Romanelli, Leaf, Greenberg and Ialongo, 2008).

6. Conceptual and Theory Background Attunement Attunement is the process of sensing and reading students’ state of being and adjusting one’s instruction accordingly ( De-Wolf and Van LJzendoor, 1997) Teachers can be trained to improve their attunement practices (Hamm, Farmer, Dadison, Gravelle and Murray, 2010; Poulsen, J., & Fouts, G. (2001) Self Determination Theory Self Determination Theory (SDT) served as a framework to to explore coaches’ responses to presenting issues among school staff (Deci and Ryan, 2000) Individuals are more likely to engage constructively when basic needs of competence, autonomy, and relatedness are met (Deci and Ryan, 2000)

7. Conceptual Model School, Teacher & Principal Characteristics Coaching Attunement Fidelity of Implementation Student Outcomes

8. Overview & Research Questions A qualitative study of interviews of 4 SEL coaches Each coach worked with three schools, 3 times per year, for 3 years (total of 9 sessions) Research Questions: Research Question 1: How do SEL coaches attune their training and support when interacting with school principals and teachers? Research Question 2: What factors lead SEL coaches to attune their work with school principals and teachers?

9. Responsive Classroom Efficacy Study (RCES) Principal Investigator: Dr. Sara E. Rimm-Kaufman, Center for Advanced Study of Teaching & Learning (CASTL), University of Virginia Three-year cluster RCT (random assignment at the school level) 24 schools in the mid-Atlantic 2000 children and their teachers from grades 2-5 Supported by the Institute of Education Sciences through grant R305A070063 to the University of Virginia

10. Methods Interviews were semi-structured, audio-recorded, de-identified and fully transcribed Coding was completed first cycle exploratory coding based on literature findings that contextual influences such as principal support, school climate, and teacher factors second cycle of pattern coding based on emergent data Data was coded by three researchers, Κ >70%, (Cortina, 1993) and analyzed using the software program Dedoose. For this study, approximately 100 codes were used

11. Results Research Question 1 Coaches attuned to principals and teachers largely by the following: (1) Deviating from the prescribed training curriculum (2) Varying the level of candidness and detail in feedback (3) Establishing personal relationships

12. Results: Example Excerpts Example of deviating from the curriculum: “I structured their meetings differently and started each session with a little time for them to express their concerns and in the other schools I did not need to do that.” Example of altering the amount of feedback: “They all want feedback. I do start at an earlier place when giving feedback to the least experienced. But really the feedback is team to team. The feedback I provide to team 1 is much different than the feedback provide to team 2.” Example of establishing relationships: “And this group , I know them personally. I know them personally to say stuff like, How’s your baby? So I think they have a lot more trust towards me. I don’t think they have had that in other situations.“

13. Results Coaches’ Attunement Practices

14. Results Research Question 2 Coaches reported attuning their training and support largely based on four teacher and school factors including: (1) Lack of teacher buy-in (2) Teacher background variance (3) Stressful environment (4) Weak leadership * Positive factors emerged via post hoc analysis: positive attitude, strong principal leadership, positive environment

15. Results: Example Excerpts Example of lack of teacher buy in: “….and she says, ‘Coach, alright, I have had enough of this. I want you to convince me that this is the way I should teach. Because I am not buying it.’” Example of teacher background variance: “She came from a special education background, so she was teaching special education the first year and now she’s got her own fifth grade classroom, so it’s a little bit of a steep learning curve for her.” Example of stressful environment: “They’re all feeling an increased pressure around time because they’re still trying to teach everything they were teaching, plus they’re trying to do this separate intervention time.” Example of weak principal leadership: “…their principal hadn’t asked them if they wanted to do this, that the principal didn’t really care…”

16. Results: Driving Factors of Attunement Practices For All Coaches LBI=Lack of Buy In TBV=Teacher Background Variance SE=Stressful Environment WPL= Weak Principal Leadership PF= Positive Factors

17. Results Coaches’ Attunement Practices LBI=Lack of Buy In TBV=Teacher Background Variance SE=Stressful Environment WPL= Weak Principal Leadership PF= Positive Factors

18. Findings & Implications Findings Attunement appears to occur naturally without specific training. Attunement practices related to the driving factors in the literature. Attunement practices appear to differ by coach, however all coaches attuned most often by deviating from the curriculum. Unexpectedly, attunement also occurs due to positive factors. Implications Findings may contribute to our understanding of how coaches respond to presenting needs of school staff members. The practice of attunement may be a strategy for coaches to use when working with schools. Training of coaches may include attunement training and motivation frameworks such as Self Determination Theory.

19. Limitations & Future Research Considerations Limitations The RC coaches of this study had on average over eight years of professional experience. This experience level may have contributed to their capacity for attunement. The data used for this study were not collected as the study progressed. The data were collected at the conclusion of a three-year time period. As a result, narratives may contain errors and misrepresentations. Future Research Considerations Future research may include a mixed methods design that includes an experimental condition that includes attunement training for coaches. This could lead to more causal based results. Future research may consider the inclusion of teacher data regarding their experiences with coaches.

20. Contact information Joseph W. Pieri, MS Doctoral Student School of Education University of Pittsburgh 230 Bouquet Street Psychology in Education 5922 W. Wesley Posvar Hall Pittsburgh, PA 15260 jwp35@pitt.edu Shannon Wanless, PhD Assistant Professor School of Education University of Pittsburgh 230 Bouquet Street Psychology in Education 5937 W. Wesley Posvar Hall Pittsburgh, PA 15260 swanless@pitt.edu

21. Implications The findings contribute to the unpacking of the nature of interactions between SEL coaches and school staff members This study has the potential to contribute to our knowledge base of how to prepare coaches and help them to follow a more empirically supported approach to their work, which could lead to improved program implementation. The results of this study suggest that practice of attunement may be an effective strategy for coaches to use when working with schools. Training of coaches may include attunement training and increased familiarization and reliance on conceptualized and operationalized engagement or motivation frameworks such as Self Determination Theory. The findings of this study have the potential to explore new pathways for understanding, improving and recommending coaching practices (Skinner and Belmont, 2012).

22. Limitations Due to the limited number of coaches interviewed and the confinement of the geographic variance among the site schools, the generalizability of the findings may be limited. The RC coaches of this study had on average over eight years of professional experience. This experience level may have contributed to their capacity for attunement. The data used for this study were not collected as the study progressed. The data were collected at the conclusion of a three-year time period. As a result, narratives may contain errors and misrepresentations.

23. Coaching Background Information Coaching efforts tend to be most effective when there are: highly engaged instructive training sessions follow up observations opportunities for specific feedback (Joyce and Showers, 2002)