School and Teacher Effects: A Team Effort

School and Teacher Effects:A Team Effort Sharon Walpole University of Delaware Michael C. McKenna University of Virginia

So how’d we do last year? Let’s look at some data from the UGA external evaluator’s report

But WHY? Think about your most effective and your least effective kindergarten teacher. They both have the same materials. They have the same professional support system. They both have the same reading block. What is it that actually differs between the two and might lead to differences in achievement?

But WHY? Think about your most effective and your least effective first-grade teacher. They both have the same materials. They have the same professional support system. They both have the same reading block. What is it that actually differs between the two and might lead to differences in achievement?

But WHY? Think about your most effective and your least effective second-grade teacher. They both have the same materials. They have the same professional support system. They both have the same reading block. What is it that actually differs between the two and might lead to differences in achievement?

But WHY? Think about your most effective and your least effective third-grade teacher. They both have the same materials. They have the same professional support system. They both have the same reading block. What is it that actually differs between the two and might lead to differences in achievement?

Lessons from School Effectiveness Literature School Differences Portion of variance in achievement explained Teacher Differences Individual Differences

An LC can form judgments about teacher differences. It’s harder to judge differences at the school level because most of us lack the opportunities to make such comparisons.

Nevertheless, factors at the school level can have strong effects on achievement. The better we understand them, the more we can control them through leadership.

Differences among teachers have a clear impact on learning. Coaching attempts to reduce some of these differences by guiding teachers toward best practice.

But what about differences among schools? Can school factors be important as well?

What are some differences among kids that affect achievement? What are some differences among teachers that affect achievement? What are some differences among schools that affect achievement?

Organize Your Hunches

Let’s look at three studies that attempted to identify school effects on reading achievement. What can we can learn from them?

Beat-the-Odds Study* Design Features Looked at school and teacher factors Used measures of word readings, fluency, and retellings 14 schools, 11 of which “beat the odds” 2 teachers at each grade, and 4 children per classroom Relied on interviews, surveys, observations, and scores School factors positively correlated with growth Forging links with parents Using systematic assessment Fostering communication and collaboration *Taylor, Pearson, Clark, & Walpole, 2000.

State-Level Outlier Study* Design Features 2 high-achieving, 1 low-achieving from 3 clusters: country, main street, uptown Used state tests, interviews, and observations School factors present in the high-achieving schools Strong leadership and commitment Teacher knowledge Time and opportunity for children to read Commitment of 8-10 years to the change process *Mosenthal, Lipson, Torncello, Russ, & Mekkelsen, (2004).

Curriculum Effects Study* Design Features Compared 4 major reform models in first grade Each used a coach, lots of PD, and regrouping of kids 4 experienced schools in each model Measures of decoding, comprehension, vocabulary; classroom observations Comparisons All of the schools were relatively successful None of the 4 models proved best *Tivnan & Hemphill, 2005

What lessons do these studies teach?

Common Characteristics • No one curriculum or intervention model is a magical solution to student achievement problems • Intense focus on the school’s goals is associated with success • Assessment, communication, collaboration • Leadership, vision, knowledge • PD plus specific curriculum

Differences From Us • These schools were successful or experienced already; we are striving to be successful and we are still new to our curricula • These schools had different demographics than we do (except for the Curriculum Effects Study) • These studies do not include the effects of intensive interventions

Limits of Generalizability • We can’t tell whether these characteristics are causes or characteristics of success • We don’t know whether these factors “transfer” to striving schools

Now let’s look at a fourth study – one with better lessons for Georgia and Reading First.

GA REA Study* *Walpole, Kaplan, & Blamey (in preparation).

So What IS Assessment-Based Planning Anyway? Schools with higher achievement were rated higher on these two characteristics: The leaders made thoughtful choices to purchase commercial curriculum materials to meet school-level needs. The leaders of this project designed a comprehensive assessment system that teachers used to differentiate instruction.

How Did We Find Out? We made a special rating sheet to rate the levels of implementation on all aspects of the project. We correlated those ratings with student achievement to find the ones that were most powerful.

How Did We Find Out? Four variables “survived” the correlations • Differentiation strategies for word recognition and fluency • Strategy and vocabulary instruction during read-alouds • Careful choice of instructional materials • School-level design of an assessment system linked to instruction

In our RF work . . . These variables correspond to: • Differentiated word recognition strategies for needs-based work • Interactive read-alouds of children’s literature (Beck + Duffy) • Use (or purchase) of curriculum materials based on their match with emergent achievement data • Selection of assessments that are really used to plan needs-based instruction

How did we find out? These 4 variables were highly correlated with one another; we combined the two leadership variables and the two differentiation variables. We controlled for LNF at January of kindergarten, we controlled for SES, and assessment-based planning was still correlated with first-grade scores at the school level.

What Do We Need to Learn? • We need to see what variables are powerful in GARF • We need to add the level of the teacher • Identify the characteristics we are targeting • Collect meaningful observational data on them to see whether they make a difference • We need to use the data we have and get the data we need

That’s a tall order! (But we think we can tackle it together.)

Why? From a design standpoint We are all trying to collect student data to measure the success of our programs. • It does not make sense to measure program effects without measuring treatment fidelity. • It does not make sense to measure treatment fidelity without observing the treatment. • It does not make sense to document treatment fidelity without trying to improve it.

Let’s look at the concept of “innovation configuration.” This is a way of finding out how fully we are implementing Reading First.

Innovation Configuration* *Hall & Hord, 2001

http://www.nsdc.org/standards/index.cfm Context

Moving NSDC's Staff Development Standards into Practice: Innovation Configurations By Shirley Hord, Stephanie Hirsh & Patricia Roy

Innovation Configuration for Teacher’s Professional Learning

Procedure for Making an IC • Designer of an innovation describes ideal implementation of various components • Those “ideals” are compared with “real” implementation through observation • The “reals” are lined up from least like the ideal to most like the ideal • Then the IC can be used for observations, and even linked to student achievement!

Here are our classroom-level “ideals” for GARF so far.

Physical Environment The classroom is neat, clean, and organized so that the teacher can monitor all children and accomplish whole-group and needs-based instruction and so that children can get the materials they need. Wall space is used to display student work and curriculum-related materials that children need to accomplish tasks.

School and Teacher Effects: A Team Effort