Attack of the Standards

1. Attack of the Standards The Standards Plus Tutoring Program formerly used as a District SES Program by Tracy Alberry EDU 735

2. Guiding Question: Are Supplemental Educational Services and after school tutoring programs, run by school districts, effective in increasing student achievement?

3. Outline of Presentation • Background and Research on Tutoring • Background Supplemental Educational Services • Proposed Studies • Description Standards Plus Program • Results of District • CST Results of XYZ Elementary • Discussion of Results • Final Recommendations

4. Merriam-Webster’s online dictionary defines to “tutor” as: 1 : to have the guardianship, tutelage, or care of 2 : to teach or guide usually individually in a special subject or for a particular purpose (http://www.merriam-webster.com/dictionary)

5. The Research • Bloom (1984) found that 80% of students who did poorly in conventional education profited from tutoring. The 1:1 feedback and correction contributed to success. • Cohen , Kulick & Kulick (1982) in their meta analysis of peer tutoring found that students who were peer tutored outperformed those not tutored on two levels: subject matter and positive attitudes toward school work (Rabow, P. xxiv, 1999).

6. The Research (cont.) • Miller (2001) proposed that educational, after-school programs emerged to decrease learning achievement gaps in students. • Growing evidence suggests that after-school program participation is associated with higher grades and test scores, especially for low-income students (Hamilton & Klein,1998; Ohio Hunger Task Force,1999; Schinke, Cole,& Poulin,1998).

7. Silver Bullet Solution • After-school programs are presented as the latest “silver bullet solution” to social and educational challenges, but successes will be questionable if positive results can not be found and measured (Miller, 2001). • If we really want after school programs to become at least a part of the solution, we need to provide them with the financing staff and knowledge base to become highly effective program (Miller, 2001).

8. Supplemental Educational Services (SES) • Since the passage of the No Child Left Behind (NCLB) Act of 2001, schools in Year 2 of Program Improvement must offer supplemental educational services to low-income students. Year 2 Program Improvement schools are schools that receive Title 1 funds and have not made Adequate Yearly Progress (AYP) for 3 consecutive years. • Supplemental educational services (SES) are tutoring services offered to low income students by outside tutoring companies or agencies.

9. Supplemental Educational Serives • According to Burch (2007) only Minneapolis and Chicago school districts have done studies attempting to assess SES’s impact on learning outcomes “Research to date offers only limited understanding of what kinds of assessment might be useful in determining the costs and benefits of various SES models” (p.15).

10. Cohen 2003 • Cohen (2003) also stated that supplemental providers needed experienced tutors and must have demonstrated success with students. Furthermore, supplemental services should have provided research-based instruction and proven strategies to help students learn. Also, each student’s program must be designed to meet his or her specific needs. Because students were diverse in their methods of learning, their programs must be customized while meeting state standards. • The No Child Left Behind Act defines supplemental educational services as tutoring and research based academic enrichment programs that supplement but do not replace-instruction provided during the school day. (Cohen 2003) and “…must be designed to increase students‘ academic achievement and help them meet state standards.” p.36

11. Cohen 1982 • Achievement In 45 of the 52 achievement studies the examination performance of students who were tutored was better than the examination performance of students in a conventional class. (Cohen 1982)

12. Problems faced: The debate continues in the research on whether or not SES services are effective. Findings vary by researcher with effectiveness ranging by provider. The problem researchers face is that there is a lack of comparable data to use to measure effectiveness. Each provider uses their own pre and post test. Standardized tests might be used, but even scaled scores from year to year can not be compared for the same student. Control groups would have to be used to measure data that used standardized test scores.

13. Proposed Study • A comparison of California Standardized Test scores in English Language Arts between students attending District run SES tutoring and students not attending tutoring, and students attending SES providers should demonstrate that District SES run tutoring is associated with Higher CST scores. In addition, the study may show that an increase in student achievement may correlate to the number of hours students attend a provider’s tutoring sessions. • An examination of pre and post test data and hours of tutoring attended may show short term achievement gains of students attending the tutoring program.

14. DESCRIPTION OF THE DISTRICT RUN SES PROGRAM *A description of other SES providers will be furnished upon further research.

15. Attack of the Standards,Standards Plus • Originally used as a District run SES tutoring program with classes containing 6-10 students. • We have adapted Standards Plus to be used during RUSD after school tutoring sessions. • Teachers should teach 1-2 math lessons and 1-2 English mini lessons per day. • Each lesson takes 10-15 minutes to teach. • *Due to the District entering Program Improvement status, the distinct is now unable to be a SES provider.

16. Language Arts and Mathematics • Explicit direct instruction lessons designed to teach discrete elements of essential California standards. • Little or no teacher preparation. • Provides multiple opportunities to teach and reteach state standards. • Includes review and maintenance lessons. • Proven effective in teaching and learning standards.

17. Goals of Instruction • Repeated exposure to the essential standards ensures equal access to the standards for every student. • Rapid pacing of instruction keeps student attention focused. • Test taking strategies are acquired by students. • Grade level content vocabulary is explained. • Increased understanding of the standards by the students, parents, and teachers.

18. Lessons are explicitly taught using direct instruction. • Not to be used as worksheets, independent work or homework. • Brisk pacing is important. • Presents experience with the standard. They are NOT for teaching mastery. • A short assessment is presented after every 4 lessons. Assessments are presented as a review lesson of the last 4 lessons. Numbers of lessons vary by content cluster.

19. TEST TAKING CLUSTER OF LESSONS • There is a special test taking cluster of lessons that can be used before the CST to maximize student performance (10 lessons for Math 5 lessons for English). • Review lessons teach the high stakes clusters as well.

20. Weekly Student Assessments Assessments include four multiple choice items with an answer box similar to scan sheets. • Assessments identify students’ understanding of the concept taught. • Identifies students for re-teach and/or enrichment.

21. Maintenance and Review Lessons: • Additional lessons are scheduled throughout the binder to reinforce concepts and skills. • Maintenance lessons located in separate section of the binder. Should be used for reinforcement. • Review lessons are lessons targeted to help students on the CST.

22. Teacher Page Template (see pink handout) • Lessons are basically made up of 5 steps: • Introduction • Guided Practice/Instruction • Independent Practice • Answers • Closure

23. Designed to attack the standards • Selects standards students have scored low on • Selects standards with the highest stakes on the CST. • Calendars are predesigned to fit each individual class.

24. Day 1 and Day 2 are in columns. Weeks go across the rows.

26. 254 students across 7 Schools Student hours attended ranged from 1 to 32 hours each.

27. Results of the Program: (Pre/Post Tests)

28. Results of the Program: (Pre/Post Tests)

29. In future extended studies: • The test score data will be examined for outlier using the standard deviation which will be found. Outliers more than 3.5 standard deviations from the mean will be removed. • The assumption of homogeneity of variance will be examined to see if the standard deviations will be within four times of each other. • The students may be grouped into two groups. Those receiving 20 or more hours of tutoring were compared to the second group of students with 0 hours of tutoring. • An independent t-test and a hierarchal regression will be run and findings examined.

30. XYZ ELEMENTARY A look at CSTs Out of 59 students who attended tutoring only 37 had valid CST scores for 2007 and 2008.

33. Discussion: • Scores fit the normal bell curve. • It is difficult to show associations between an effect and an outcome when students may be receiving outside effects. • It is difficult to use 2007 CST scores to predict 2008 CST scores or measure growth because of the apple orange effects of scaled scores. • Data needs to be gathered from a control group. • WHEN WILL THE POWERS THAT BE ACTUALLY MAKE IT EASY FOR US TO MEASURE ACHIEVEMENT??

34. Miners 2007 • Miners (2007) in “SES Effectiveness Is a Matter of Debate” reviewed two recent reports on SES with differing opinions using data from different sources. One reported SES is giving gains in student achievement while the other reported evidence is inconclusive due to states’ lack of monitoring capabilities. It shows how data from different sources can be used to see two completely different outcomes. The article highlights states abilities to monitor SES programs. The article failed to go in depth regarding the data used in the studies.

35. Sunderman (2006) • Sunderman (2006), in “Do Supplemental Educational Services Increase Opportunities for Minority Students” goes so far as to argue that supplemental services actually are detrimental to schools. It makes the point that there is scant research on the effectiveness of SES programs. The article also claims that money used for SES services would be better off spent elsewhere. The article overall offers subjective opinions and not enough of objectives facts, though he lists a long list of references; his article’s finding may be open to debate.

36. Burch (2007) • There seems to be no consistency between states on implementing and monitoring SES programs. The policy brief by Patricia Burch (2007) called “Supplemental Services Education Services under NCLB”, examined several studies and national information to make recommendations for SES, which is as of 2007, is entering its sixth year of implementation. • Highlighted are the facts that states and districts often face limitations such as lack of funding, in implementing and monitoring the effectiveness and policies of SES programs. Patricia Burch (2007) recommended the NCLB law be redesigned.

37. Additional Studies • I would recommend a mixed methods study that incorporates observations, interviews and surveys from parents and student. • Parent survey was done for SES but not the Standards Plus Tutoring. Out of 600 surveys mailed, only 73 were returned. Surveys were mailed in English and Spanish.

38. Parent Survey of SES

41. Monitoring Forms Standards Plus • SCHOOL: ___________________________________________ • Date: ___________ Visited by: ____________ • Teacher: _______________________ Classroom #: _________Grade _________ • Number of Students: _____________ Lesson/Standard being taught________________ • ________________________________________________________________________ • Follow up: ______________________________________________________________________ • ______________________________________________________________________ • ______________________________________________________________________

42. Monitoring Forms SES

44. Final Recommendations & Questions • A uniform procedure such as a uniform pre and post test for tutoring providers needs to be implemented that would allow for student achievement to measured in a meaningful way. • If achievement can not be measured, this may not be the best use of time and funds. • A closer look at what constitutes effectiveness is needed. For instance would learning study skills be more effective in increasing achievement than in study math? • Does school tutoring by teachers have a greater impact than private companies tutoring?