Eighth Grade Access to Algebra I: A Study of Virtual Algebra Peggy Clements, Education Development Center, Inc. With s

1. Eighth Grade Access to Algebra I: A Study of Virtual Algebra Peggy Clements, Education Development Center, Inc. With study team members Jessica Heppen & Teresa Duncan (AIR), Cheryl Tobey & Katherine Culp (EDC)

2. Regional Educational Laboratory Northeast and Islands (REL-NEI) http://relnei.org

3. Why a Study of Online Algebra I for Eighth Graders? • Why Focus on Algebra I in 8th Grade? • Student achievement in mathematics a national and regional concern • Policymakers advocate “pushing down” Algebra I to 8th Grade because • Algebra I is a gatekeeper course • Taking Algebra I in 8th grade places students on a trajectory toward advanced mathematics and science course sequences

4. Why a Study of Online Algebra I for Eighth Graders? • Why study an online course? • Access to Algebra I in 8th grade is not widespread and online delivery may be an effective way of broadening access • Offering Algebra I is especially difficult for rural schools • Use of online courses in K-12 settings is growing rapidly • 40,000 to 50,000 students in 2001-02 • 520,000 students in 2004-2005 (McLeod et al, 2005) • Over 1 million students in 2007-2008 (Picciano & Seaman, 2009) • Need for rigorous research to establish effectiveness

5. Eighth Grade Access to Algebra I • Designed to determine the impact of offering Algebra I through an online course to 8th graders who would not otherwise have access to the course • Not a comparison of online versus a traditional course, but whether offering an online Algebra I course is beneficial in schools where Algebra I is not currently offered to 8th grade students • Student outcomes include short- and long-term measures of math achievement

6. Context for the Study • Schools in Maine & Vermont that serve 8th graders and do not typically offer a full Algebra I course to 8th grade students • Most schools are rural, some located in very remote locations • 70 schools (50 in Maine, 20 in Vermont) • Average number of 8th graders per school: 29.7 • Range: 1 to 146 8th graders • Half of the schools are very small (serve 16 or fewer 8th graders)

7. Context for the Study • Primary focus on “algebra-ready” (AR) 8th graders • On average schools identified 23% of their 8th grade cohort as “ready” to take algebra • Does online Algebra I benefit these students? • Secondary focus on non-“algebra-ready” (N-AR) 8th graders • Does offering Algebra I online to AR students have any unintended consequences for N-AR students, who remain in the regular math class?

8. Policy Question This study is designed to generate policy-relevant evidence to answer the question: • In schools that do not typically offer Algebra I to eighth graders, is it beneficial to offer Algebra as an online course? ~ Where “benefit” is assessed in terms of student achievement in mathematics and subsequent high school course-taking, and the delivery of Algebra I online is to 8th graders considered “AR” by their schools.

9. Primary Research Questions What is the impact of offering Algebra I online to AR students on their • end-of-8th grade algebra achievement? • subsequent high school course-taking? ~ The direct impacts on AR students are of primary concern for assessing whether this intervention is an effective way for schools to broaden access.

10. Secondary Research Questions What is the impact of offering Algebra I online (to AR students) on • N-AR students’ mathematics achievement (algebra and general mathematics)? • N-AR students’ subsequent high school course-taking? • AR students’ general mathematics achievement? ~ The indirect impacts on N-AR students are of secondary concern for assessing whether there are an unintended consequences of using this intervention to broaden access.

11. Research Design • Randomized Control Trial (RCT) • Schools randomly assigned to treatment or control • Treatment – school receives online Algebra I course for their AR students during SY2008-2009 • Control – school conducts “business as usual” during SY2008-2009 • All schools identified their AR students prior to random assignment

12. Treatment versus Control School Year 2008-2009 Business as usual – 8th Grade Math Algebra I Online Algebra-ready Algebra-ready ALL 8th GRADERS ALL 8th GRADERS Non Algebra-ready Non Algebra-ready Treatment Control

13. Student Outcome Measures • Mathematics achievement • Mastery of algebraic concepts test – end of 8th grade • General mathematics achievement test – end of 8th grade • High School Course-taking • High school mathematics and science course-taking (AR students only)

14. Implementation of Intervention Algebra I Course Structured lessons and activities- presentations, practice exercises and written assignments Online Self-paced However, over the course of the year students were expected to complete the entire course Asynchronous (anytime, any place: students do not need to be online at the same time as the teacher) Online teacher On-site proctor to monitor students taking the online course

15. Implementation of Intervention Online Teacher Responsible for instruction through online courseware and communication tools Grades assignments and assessments Answers questions through discussion boards and internal messaging system Communicates with students and onsite proctors by reading and responding to messages

16. Implementation of Intervention School-Based Proctor Supervises students during the class period Provides a supportive and structured working environment Provides the link between the online teacher, the student, local school officials, and parents as necessary (acts as the eyes and ears of the online teacher)

17. Algebra I Course Implementation • 35 Schools Received the Intervention

18. School-Based Implementation Decisions • Identifying Algebra Ready Students • School-Based Proctor • 8th Grade Math Teachers • Other classroom teachers • Principals • Gifted & Talented Teachers • Educational Technicians • Technology Teachers

19. School-Based Implementation Decisions • Classroom Location • Classroom • 8th grade math class monitored by teacher who was simultaneously teaching regular 8th grade math course to • Instructional space • proctored by staff assuming other responsibilities (e.g. GT, Principal, Computer Lab) • proctored by Educational Technicians

20. Reporting of Results • Final report scheduled for winter 2010/2011 • Conclusion of whether intervention is “effective” based on combination of results: • Positive impacts on AR students’ algebra scores or the likelihood that they go on to participate in advanced course sequences • Non-negative impacts on AR students’ general mathematics scores • Non-negative impacts on achievement and course-taking outcomes for N-AR students

21. For more information • http://relnei.org • or email me: pclements@edc.org Thank you!

22. Content and Structure of the Online Course • Two “semesters” • Algebra IA • Algebra 1B • The course is divided into units and lessons

23. Lessons- Learning the Content Interactive Textbook

24. Lessons- Learning The Content Text with Audio Support

25. Lessons- Learning The Content Text with Chalkboards

26. Lessons- Learning The Content Mini Lessons – “Chalk talks”

27. Assignments Practice Exercises

28. Assignments Interactive Applets

29. Assignments Guiding Questions- Applets

30. Assignments Open Ended Prompts

31. Assessments Quizzes and Tests (Password Protected)

32. Analytic Approach Two-level hierarchical linear models (HLMs) with students as Level 1 and schools as Level 2 Basic model will capture difference in student outcomes between intervention and control schools, controlling for pretest scores, student characteristics, and school characteristics Primary analyses will estimate the impact of the intervention on AR student outcomes Secondary analyses will estimate the impact on N-AR student outcomes