Karen Abney- Fayette County Public Schools & Eastern Kentucky University

1. Get with the Program: A multivariate look at standardized test scores among Reading First schools in Kentucky. Karen Abney- Fayette County Public Schools & Eastern Kentucky University Jennifer Chambers-Mercer County Public Schools & Eastern Kentucky University Dr. Jerry Johnson, Eastern Kentucky University NCPEA Conference July, 2008 San Diego, CA

2. Reading First Grant Key Points • $500 million dollars is distributed to states, districts, and schools through competitive awards for up to 6 years to support efforts to teach literacy and increase reading development of K-3 students (particularly low-income students). • grantees must choose a “scientifically-based” reading program (SBRR) from a list of several identified by the federal government as having met their evaluative criteria for each of the three tiers of instruction • a school coach must be provided • parental involvement must be incorporated into the plan • a minimum of 90 minutes per day must be devoted to literacy • common assessments must be used (chosen by the state) • All K-3 teachers, as well as special education teachers, must receive 80 hours of Professional Development in reading per year. • Kentucky schools receive approximately 11 million dollars per year for the duration of the grant. • 74 Schools received the grant and began implementation in 2004-2005

3. Kentucky Reading First Assessments • Dynamic Indicators of Basic Early Literacy Skills test (DIBELS) • minute long assessment-recording accuracy and rate of students on initial sound recognition and production, letter naming, phoneme segmentation, and on-level passage reading. • designed to identify early reading difficulties using norm referenced benchmarks in the fall, winter, and spring testing windows. • Designed for progress monitoring students between testing windows to measure growth or the lack there of. Group Reading Assessment and Diagnostic Evaluation (GRADE) • standardized norm referenced, test. • assesses skills in phonemic awareness, phonics, vocabulary, and comprehension dependent on the grade level and age-appropriateness. • Given 3 times a year-fall, winter and spring. • Results are reported to students, parents, SBDM’s, District offices, State Coaches, KDE, Federal Reading First Administrators • All KY Reading First schools use these two tests • GRADE is considered the more important test of the two because it measures more skills overall and is similar to another standardized test which is given in all third grades across Kentucky.

4. Key Points of GRADE Results for All Kentucky Reading First Schools from 2004-2007 • The state average for K-3 students scoring 50th percentile and above rose from 30.1% to 55.0% for the first year of implementation. • It rose even further to 63.3% for 2005-2006 • 2006-2007, KY RF schools reached 71.2% of students reading at or above grade level

5. Brief Overview of Results From Previous Study- “Who’s Putting Reading First in Kentucky?” A multivariate regression analyses was conducted to investigate the extents to which school, district, and community characteristics are associated with performance levels on reading assessments. • socioeconomic status • students’ race/ethnicity • students’ special education status No significant indicators were found signifying the need for further study.

6. Literature • Controversy over the origin and implementation of the Reading First grant • Lack of peer-reviewed research about results • Some government and other organizational reports

7. Research Questions and Data • Research questions: • 1. Is reading performance related to the instructional program selected by the school? • 2. Is reading performance influenced by the length of the daily literacy block? • 3. To what extent are school-level characteristics and overall school achievement predictive of three year gains in reading performance levels in schools receiving federal Reading First grants? • Data sources: KY Dept. of Education, all Kentucky Reading First schools for which all data was available (n=71).

8. Variables • Dependent Variables: • BOY 2004 percent students reading on level per GRADE test • EOY 2007 percent students reading on level per GRADE test • Three year GRADE gain score (SY 04-SY 07) • Independent Variables: • Size • SES • Race/ethnicity • Speech and Language • Length of literacy block • Selected reading program

9. Research model • Preliminary analyses and data preparation: • Tests for skewness & kurtosis • Variables centered (per Cronbach, 1987) • Primary analyses • Descriptive Statistics • Independent Samples T-Test • One-way ANOVA • OLS regressions using SPSS 16.0

10. Descriptive Statistics

11. Independent Samples T-Test

12. ANOVA Results

13. Regression Results

14. Study Results • There were no significant differences for reading performance among schools using the different instructional programs (note: schools did differ significantly by group prior to implementing programs). • There were no significant differences for reading performance based on length of literacy block (note: only seven schools reported having 90 minutes and there were inconsistencies in the reporting of block length). • Poverty and enrollment both influence gain scores. Schools with higher poverty levels are making higher gains. Schools with lower enrollment are making higher gains.

15. Limitations • Lack of Mobility variable • School unit of analysis • Length of the literacy block variable (n cases in 90 minute category; data inconsistencies) • Delimitation to a single state (Kentucky)

16. Recommendations • Consider the potential for smaller enrollment size as policy option • Recognize accomplishments of higher poverty Reading First schools in Kentucky • Consider policy options for continuing and expanding practices implemented in Reading First schools • Use of student-level data to investigate interactive effects.

17. Contact info karen.abney@fayette.kyschools.us jennifer.chambers@mercer.kyschools.us jerry.johnson@ruraledu.org