“Don’t Use a Tablespoon to Measure Temperature” - W. James Popham, 2004

1. “Don’t Use a Tablespoon to Measure Temperature” - W. James Popham, 2004 The Importance of Using Instructionally Supportive, Standards-based Accountability Tests

2. TWIX Group Heather Batchekler Bonnie Erickson Gulnora Hundley Tyson Kuch Earney Lasten David Norman Julie Pepe Richard Reardon Scott Smith Jennifer Tobias Nicole Vaccaro Affirmative Case for Standardized Testing

3. Affirmative Proposition • “Standardized testing is a useful tool in educational systems and should be continued to be used as a tool to access a student’s competency in different areas”

4. Definition of Standardized Testing • Standardized testing is any test that is used across a variety of schools or other situations. Designers of such tests must specify a discrete correct answer for every question. This type of test includes both achievement (which measures knowledge already known) and aptitude (which attempts to predict future performance or potential) tests given to grade-school students. • Source Wikipedia-On-line Encyclopedia http://en.wikipedia.org/wiki/Standardized_testing

5. How Many Students (Kindergarten through College) Are there in the US? • Almost 73 million

6. This is a very large number of people (73 million) who must be assessed annually.This would be impossible without standardized testing

7. The system is not perfect, but it is the only effective way to manage the numbersStandardized testing levels the playing field and allows “fair” and equal comparisons

8. In the next twenty minutes the Twix team will show you: - Other testing methods are subjective/not an equal comparison -Share success stories for standardized testing -How we believe standardized testing should be used /administered - Focus on standardized testing as a useful tool and not “high stakes” testing

9. We are also going to take some of the common arguments against standardized testing that our group came up with and address them individually.

10. So we ask you to keep this question in mind as both groups make their presentation… What other measurement methods are available that are: 1. Subjective 2. Equal 3. Effective 4. Able to be administered to 72 million students annually

11. Benefits of Using Instructionally Supportive Standards-Based Accountability Tests • Supply accurate accountability information for school evaluation • Help teachers do a better instructional job • Provide student-level diagnostic information and per-standard status

12. What Does This “Magic” Test Look Like? • Assessment based on relatively small number of challenging academic content standards. • Custom developed assessment used to measure each child’s mastery of content standards. • Assessment provides standard-by-standard reports to children, teachers and parents. • Assessments have clear descriptions of content standards being assessed.

13. Should We Evaluate Schools Using Standardized Tests? • Instructionally Supportive Accountability Tests • Measure significant standards • Have clear descriptions of what is assessed • Provide standard-by-standard mastery reports • Provide instructional information for teachers

14. How Should We Evaluate Schools? • Student work samples that reflect student mastery of demanding, high-level skills • Narrative essays • Oral reports • Science Fair projects • Written solutions to complex math problems • Properly collected and systematically scored

15. How Should We Evaluate Schools? • Affective Data • Inventories • Attitudes and Interests • Self-Reported, Impact on Groups • Clarifies Quality of Instruction • Preinstruction to postinstruction

16. How Should We Evaluate Schools? • Non-test Academic Indicators • Attendance rates • Tardiness Statistics • School-specific Data (graffiti, graduation rates)

17. Commission on Instructionally Supportive Assessment2001 Report Experts identified suggestions regarding how NCLB assessment would enhance not erode, educational quality They identified assessments that would: *Provide score-based accountability information needed to evaluate school quality *Help teachers improve instruction www.aasa.orgwww.naesp.orgwww.principals.org www.nea.orgwww.nmsa.org

18. Consider… • Wheelan and Kesserling (2005) found that the level of faculty collaboration influenced student outcomes on standardized tests especially in low socioeconomic urban and rural venues. Results indicated that staff size, location and district poverty level had significant impacts on student performance. Wheelan, S. & Kesserling, J. (2005). Link between faculty group development and elementary student performance on standardized tests. The Journal of Educational Research, 98(6), 323-330.

19. Consider… • Students in urban and rural districts with an elevated poverty level scored at a higher level on standardized tests within a highly collaborative school

20. Consider… • The results of this study suggest that facilitating faculty collaboration under the guidance of educational consultants will increase student learning and student performance on standardized tests in even the most impoverished rural and urban settings. • Involving the faculty in this process creates a sense of ownership

21. Does Testing Really Improve Student and School Performance? • Decesare (2002) states that standardized testing has been used in education to sort students into vocational or college preparatory tracks. Minimum competency testing was used in the 70’s and 80’s in order to determine eligibility for graduation. • During the last 15 years most states have created standards for students in each grade level in order to improve the educational system and challenge students academically. Decesare, D. (2002). How high are the stakes in high-stakes testing?Principal (Reston, VA), 81(3), 10-12.

22. Does Testing Really Improve Student and School Performance? • High-stakes testing is now being utilized to assess students, schools and the personnel within the school system. In many cases promotion to the next grade level and/or graduation depends on the student’s scores on standardized tests. Student’s test scores are also used as one factor in determining a schools grade or level of performance. • Schools performing below a certain standard may receive a warning, be placed on probation or be labeled a “failing’ school. Chances to remediate are given to these schools with varying levels of support from state departments of education.

23. Does Testing Really Improve Student and School Performance? • The author maintains that testing serves the purpose of improving student and school performance. Decesare, D. (2002). How high are the stakes in high-stakes testing? Principal (Reston, VA), 81(3), 10-12.

24. Does Testing Really Improve Student and School Performance? • A little over a decade ago Lewin and Lu (1990) reported that widespread testing in China led to low-level recall, discouraged creativity and originality, narrowed the content and framing of curricula, elevated content over skills, furthered the credentialist spiral, and encouraged rote learning' (As quoted in Morrison and Tang Fun Hei Joan, 2002) • Morrison, Keith, & Tang Fun Hei Joan (2002). Testing to destruction: A problem in a small state. Assessment in Education, 9(3).

25. Does Testing Really Improve Student and School Performance? • When alternative assessment is the only assessment used, parents will argue for not knowing how their child stands compared to everyone else (Haladyna, 1999). • Haladyna, Thomas M.  A complete guide to student grading. Boston: Allen and Bacon. 1999

26. The Case For… • According to Wilmore and McNeil (2002), in a paper presented at the Annual Meeting of the American Educational Research Association, there is a “correlation between the GRE, gender, race, and undergraduate GPA” in predicting academic success. Wilmore, E. L., & McNeil, J. J., Jr. (2002, April). A Five-Year Analysis of GRE, Race, Gender, and Undergraduate GPA as Predictors of State Certification Examination Resultspresented at the 2002 Annual Meeting of the AmericanEducational Research Association. Retrieved September 11,2005, from http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000b/80/0d/e1/ec.pdf

27. The Case For… • In a study by Black and Wiliam, failure on assessment is linked back to teachers…Black and Wiliam found: • Students are not able to assess themselves to prepare for formative assessments because teachers often don’t give the pupils a clear picture of the objectives. • Many teachers want students to develop an understanding of material, however their tests encourage rote and superficial learning instead. • Primary teachers often tend to emphasize quantity of work and “neglect quality in relation to learning” • The giving of marks and the grading function are overemphasized, while the giving of useful advice and the learning function are underemphasized. • - Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappa, 80(2), 141-143.

28. The Case For… • According to Wilmore and McNeil (2002), in a paper presented at the Annual Meeting of the American Educational Research Association, there is a “correlation between the GRE, gender, race, and undergraduate GPA” in predicting academic success. Wilmore, E. L., & McNeil, J. J., Jr. (2002, April). A Five-Year Analysis of GRE, Race, Gender, and Undergraduate GPA as Predictors of State Certification Examination Resultspresented at the 2002 Annual Meeting of the AmericanEducational Research Association. Retrieved September 11,2005, from http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000b/80/0d/e1/ec.pdf

29. The Case Against… • Teachers felt rushed to fit presentations in for all the content required in high-stakes tests to the point they would sacrifice quality of instruction, just to get all the content presented in a study published by Intervention in School and Clinic. • Mastropieri, M. A., Scruggs, T. E., Greaetz, J., Norland, J., Gardizi, W., & McDuffie, K. (2005, May). Case studies in co-teaching in the content areas: Successes, failures, and challenges. Intervention in School and Clinic, 40(5), 260-270

30. The Case For… • In a research study of 214 undergraduate students, a composite final examination covering the curriculum recommended by the General Medical Council “achieved an overall reliability sufficient for high-stakes decisions on student clinical performance” when questions were properly weighted. • Wass, V., McGibbon, D., & Van der Vleuten, G. (2001). Composite undergraduate clinical examinations: how should the components be combined to maximize reliability? Medical Education, 35, 326- 330.

31. The Case Against… • For references that cite the AERA, APA, and NCME standards that a single test score should not be used for a high-stakes decision: “In the January 7, 2000 ruling by U.S. District Judge Edward C. Prado (GI Forum et al. v. Texas Education Agency et al., 2000), upholding the Texas Education Agency’s (TEA’s) use of the Texas Assessment of Academic Skills (TAAS) as a high school graduation requirement,” the court found “the AERA, APA, and NCME (1999) standard that a single test score should not be used for a high-stakes decision is not violated if multiple opportunities exist for the student to pass.” (personal note: this is probably because multiple test sittings = more than one test score, and therefore does not fall under the single test score standard). • Schafer, W. D. (2000). GI Forum v. Texas Education Agency: Observations for States. Applied Measurement in Education, 13(4), 411-418.

32. Use vs. Misuse • Holloway regards standardized testing as a tool that will bring about long-term improvement in the educational system provided the tests are evaluated and utilized correctly Holloway, J. (2001). The use and misuse of standardized tests.Educational Leadership, 59(1), 77-79

33. Use vs. Misuse • A survey conducted by Hart and Teeter (as cited in Holloway, 2001) indicated that half of those surveyed thought that standardized testing is beneficial in regard to educational improvement. • One third of the respondents were concerned about the potential for misuse of standardized tests Hart, P. & Teeter, R. (2001) A measured response: Americans speak on ducational reform. Princeton, NJ: Educational Testing service

34. Use vs. Misuse • Robert Linn (as cited in Holloway, 2001) provides guidelines for the use of standardized tests: • include all students in the assessment process • do not rely on a single test, emphasize year to year performance rather than comparing schools, • be aware of measures of uncertainty in tests and evaluate the tests and testing procedures diligently. Linn, R. (2000). Assessments and accountability. Educational Researcher, 29(2) 4-17

35. Success Stories – Achievement Tests • Caliente Elementary School – Making dreams a reality • Students with Disabilities • – Dylan’s story • SMART and THRIVE Programs – Technology programs in some Little Rock, Arkansas Schools • Capital City - An innovative charter school

36. International Testing

37. France 7 part levels UK 7 part levels

38. Germany 11 part levels Japan 5 part levels

39. Italy 8 part levels Canada 5 part levels

40. Russia 6 part levels USA 5 part levels

41. G 8 Countries (Calculation basedon ISCED* levels) *International Standard Classification of Education Source: UNESCO

42. Besides the Education System, Who Uses Standardized Testing Successfully? • CPA’s BAR Professional organizations/certifications

43. Summary- Benefits • Guarantee a minimum level of knowledge • Standardize learning • License workers • Global understanding • Allows for group comparison • Objective • Reliable and valid

44. Alternative Assessment • Can be worse than traditional • Subjective • Testing cannot be replicated • Group results not available • Global Comparison not applicable • Labor intensive • Modified teaching practices

45. Alternative School • Restructured school day • Longer class periods • Strong advisement system • Same group of students and teachers • Small number of students per teacher • Outside meetings with parents • Mandatory community service

46. “Layered on top of traditional school structure, curriculum mandates, texts, and teaching practices, these assessment strategies would fail, as would the teacher and students engaged in using them.” -Authentic Assessment in Action by Linda Darling-Hammond, Jaqueline Ancess and Beverly Falk.

47. Key Ideas • Standardized testing is not the same as high-stakes testing. • The test results are separate from the test. • Results of any assessment can be misused. • We live in a global world with lots of people and a need to make comparisons. • Standardized testing skills are life skills.