Online Statewide Assessment: Coming Soon to a School Near You!

1. Online Statewide Assessment: Coming Soon to a School Near You! David Judd Director; Office of Systems, Psychometrics & Measurement ResearchBureau of Assessment and Accountability April 30, 2013 MERA Conference

2. Today’s Topics Student Voice Kindergarten Entry Assessment Statewide assessment overview Next-generation assessment accessibility Statewide systems supporting online assessment & reporting Technology readiness Fall 2013 Online Testing Opportunities

3. MEAP Social Studies Online PilotStudent Comments “I enjoyed taking the online test a lot more” “It was very fun” “Make the math one too!” “I would like to use computers for the MEAP again” “I LOVED TAKING IT ONLINE !!!!!!!!!” “I feel like it was better to take it online” “I wish we took all MEAP tests online, it was faster” “The online MEAP test was a good experience!!!!!” “Can we take all the MEAP tests online?”

4. MEAP Social Studies Online PilotStudent Comments “The computer tend to make me tired and sleepy” “I did not like how I could see other computers and I would see how far (the other students) were.” “Make the words bigger” “It could use some more color” “You should tell the kids their score at the end” “Fix the highlighter tool!”

5. MEAP Social Studies Online Pilot Online or Paper?

6. MEAP Social Studies Online Pilot What distractions did students face?

7. K-2 Cognitive LabsParticipation • 18 Second Grade Teachers • 29 First Grade Teachers • 22 Kindergarten Teachers • Feedback focused on… • Overall sentiment • Technology-related issues • Test content and format • Implementation issues

8. K-2 Cognitive LabsFeedback • Teachers need to… • prepare students with computer skills • determine logistics of administering an online assessment to a large group of students • Some elements were difficult for students… • Scrolling • Drop-down boxes • System needs to… • highlight words as they are being read • allow student to turn audio off

9. K-2 Cognitive LabsFeedback Higher student engagement Acknowledges different learning styles Human voice audio was very helpful Reduction in test anxiety Necessary life skills

10. Kindergarten Entry Assessment (KEA) • Legislation provided $3.25 million to implement a KEA • Requires PD for educators • Must include a system for data entry and integration with P-20 LDS • Spotlights conducted fall 2012 to understand state of the art • Request for Proposals released March 2013 • Significant pilot testing expected fall 2013

11. The Assessment Challenge ...to here? Common Core State Standards specify K-12 expectations for career and college readiness All studentsleave high school career and college ready ...and what can an assessment system do to help? How do we get from here...

12. Transitions • MEAP SBAC (Smarter Balanced Assessment Consortium) • MME SBAC • MEAP-Access (for students with disabilities for whom both MEAP/MME and MI-Access are inappropriate)  SBAC • MI-Access (for students with significant cognitive disabilities) DLM (Dynamic Learning Maps) • Functional Independence (for students with mild cognitive disabilities) • Supported Independence (for students with moderate cognitive disabilities) • Participation (for students with severe cognitive disabilities) • Michigan English Language Proficiency Assessment (ELPA)  WIDA/ASSETS (World Class Instructional Design and Assessment)

13. Additions • Interim Assessments • Optional & Online only • Primarily grades and content areas not covered by a consortia • Science • Social Studies • MMC courses • K-2 ELA & Mathematics • Model assessment (e.g., Arts) • Kindergarten Entry Assessment

14. SBAC - A Consortium of States • 25 states representing 40% of K-12 students in U.S. • 21 governing, 4 advisory states • Washington state is fiscal agent • WestEd provides project management services

15. A Balanced Assessment System Summative assessments Benchmarked to career and college readiness Teachers and schools have information and tools they need to improve teaching and learning Common Core State Standards specify K-12 expectations for career and college readiness All students leave high school career and college ready Teacher resources for formative assessment practices to improve instruction Interim assessments Flexible, open, used for actionable feedback

16. A Balanced Assessment System Two types of Interim Assessments - Summative Clone (mimics accountability test blueprint) - Interim Cluster (deeper dive into assessment targets)

17. Smarter Balanced Sample Items • October 9th, 2012 release of online tool holding 50+items • Mathematics • ELA • Samples from all 7 grades (3-8 and 11) • Item type conversations • New level of rigor expected by the Common Core State Standards

18. Smarter Balanced Sample Items

19. Smarter Balanced Sample Items

22. Performance Tasks • New item type for Michigan • Measure concepts other item types do not cover very well (e.g., research) • Up to an estimated 125 minutes needed • Performance Tasks may include a Classroom Activity • Six sample Performance Tasks • Teacher Preparation & Resources • Specifications • Scoring Rubrics

23. Smarter Balanced Pilot Test – Spring 2013 • Scientific Sample • Over 700 Michigan schools agreed to participate • Student survey responses being collected • Volunteer Schools (Practice Tests) • Grades 3-8 and 11 in ELA and Mathematics • Similar to planned assessment experience • Available May 29, 2013 and through the 2013-14 school year • Practice Test Accommodations • Text-to-speech • Item-level pop-up Spanish glossaries for construct irrelevant terms (math tests only) • Braille • American Sign Language (ASL)

24. Smarter Balanced Pilot Test – Spring 2013 • Lessons Learned • Item development • Systems interoperability • Scoring needs • Translation methodology

25. Smarter Balanced Next Steps • Item development for spring 2014 field test • Hire a scoring service provider and build Artificial Intelligence (AI) scoring engine • Continue developing: • Computer Adaptive Testing (CAT) specifications • Reporting system • Digital library • Interim system

26. Find Out More Smarter Balanced can be found online at: SmarterBalanced.org

27. DLM – Dynamic Learning Maps 27

28. DLM Timeline

29. The Common Core Essential Elementsare specific statements of the content and skills that are linked to the Common Core State Standards grade level specific expectations for students with significant cognitive disabilities.

30. Common Core Essential Elements: Design Priorities • Define essential differences from grade to grade in • cognitive demand • content knowledge • skills-based expectations • Identify the key elements essential for each grade level. • Not necessarily a one-to-one relationship with Common Core State Standards • Align Essential Elements across and between grades

31. Third Grade Mathematics Standards: Standards: Operations and Algebraic Thinking

32. Fifth Grade English Language Arts Standards: Reading (Foundational Skills)

33. Technology Features for DLM Touch screen technology Audio via sound files American Sign Language video Pop-up context-dependent dictionaries/glossaries Text and image magnification On-screen note taking Color overlays IntelliKeys™ keyboard accessibility Masking Text readers Refreshable braille displays

34. As a result you can expect some changes in the future… Accommodations and selection process Development of Personal Needs Profiles Move towards consortium developed Participation Guidelines Accommodations Monitoring

35. ASSETS – Assessment Services Supporting ELs through Technology Systems (ASSETS) http://assets.wceruw.org/

36. ASSETS Timeline

37. Alternate ACCESS • Currently available • K (Future availability) • Grades 1-2 • Grades 3-5 • Grades 6-8 • Grades 9-12 • Designed for ELs who are also SWDs • Based on alternate performance indicators and alternate proficiency levels • Listening, Reading, Writing, and Speaking • Michigan plans to offer in 2013

38. Technology Readiness Progress Bruce Umpstead, OEII & Dave Judd, BAA Spring 2015, online assessment to the extent possible Smarter Balanced and Michigan accountability assessments will have paper/pencil version at least through spring 2017 Michigan interim assessments are optional so no paper/pencil version

39. Accessibility and Online Testing • Technology-based assessment platforms offer new opportunities and ways for accommodations to be provided to students who need them • Requires a new look at defining • Accommodations • Embedded supports/tools • Good testing practices

40. Good Testing Practices Teachers must take care to ensure students have opportunities to become familiar with technological aspects of online testing Allow time for practice tests using planned assessment platform Provide practice time using accommodations and embedded tools Begin to use/increase the use of technology in instruction Evaluate use of tools/accommodations

41. System Interfaces Local to State

42. State Systems CEPI Systems MDE System

43. BAA Secure Site Interfaces

44. BAA Online Test Delivery Engine Interfaces

45. BAA Reporting System Interfaces

46. TECHNOLOGY READINESS What we’ve learned about

47. Device Readiness • Device Specifications • Most devices meet minimum requirements • Form factors supported: • Desktop • Laptop • Netbook • Tablet • Number of Devices • Logistical challenges with limited testing devices • Pilot opportunities allow dry-run of logistics

48. Network Readiness • Speed/Capacity • Actual speed varies continually • Current utilization levels need to be considered • Reliability • Wireless access points potentially over-utilized • Consider age and performance of networking equipment

49. Don’t Feel Ready for Online Testing? Paper and Pencil versions of Summative tests will be available through Spring 2017 So, why make the shift to online testing now?

50. Advantages of Online Testing Retake opportunity More informative reports Quicker reporting No materials to inventory, monitor, and return Optional interim assessments online only Adaptive testing capability More efficient scoring Increased student engagement Improved security