Building Consensus on Effective Instructional Strategies for English Language Learners with Disabilities

1. Building Consensus on Effective Instructional Strategies for English Language Learners with Disabilities July, 2007 Martha Thurlow and Kristi Liu

2. NCLB State Assessments State assessments must provide for the participation of all students, including students with disabilities or limited English proficiency

3. No mention of English language learners with disabilities! These students are clearly included in NCLB’s reference to “all students”. They need to be included in grade-level, standards-based content instruction along with their peers.These students are clearly included in NCLB’s reference to “all students”. They need to be included in grade-level, standards-based content instruction along with their peers.

4. What Do We Know About ELLs with Disabilities? Population 357,325 in 2002-2003 (9% of ELLs) Numbers increasing with rapidly growing population More ELLs instructed in mainstream courses taught in English -- tendency increased for students with disabilities According to a recent national survey—Descriptive Study of Services to LEP Students– funded by OELA, there was a 72% increase nationwide in the number of ELLs K-12 between 1992-2002 76% of all ELLs in the country were Spanish-speaking In that same time period there was a 22% increase in the number of ELLs receiving instruction only in English. And there was a decrease in the percentage of students receiving ESL/Bilingual education servicesAccording to a recent national survey—Descriptive Study of Services to LEP Students– funded by OELA, there was a 72% increase nationwide in the number of ELLs K-12 between 1992-2002 76% of all ELLs in the country were Spanish-speaking In that same time period there was a 22% increase in the number of ELLs receiving instruction only in English. And there was a decrease in the percentage of students receiving ESL/Bilingual education services

5. Simply putting ELLs with disabilities in the mainstream classroom with standards-based content does not guarantee they learn the material!

6. Test scores show limited achievement for these students These data are old, but many states don’t have publicly available data analyses showing the achievement of ELLs with disabilities. We chose to refer to some data that we ran ourselves several years ago. Even though the data are old, it’s very possible that achievement gaps like these are still happening for this group of kids. Seeing these kinds of assessment data for ELLS with disabilities we wondered whether and how these students are being provided access to the grade-level standards-based curriculum that precedes the tests. That thought led to our grant project. This comes from the following NCEO report Liu, K., Thurlow, M., Barrera, M., Guven, K. & Shyyan, V. (2005). Graduation exam participation and performance (2000-2001) of English language learners with disabilities (ELLs with Disabilities Report 3). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/ELLsDisReport3.html These data are old, but many states don’t have publicly available data analyses showing the achievement of ELLs with disabilities. We chose to refer to some data that we ran ourselves several years ago. Even though the data are old, it’s very possible that achievement gaps like these are still happening for this group of kids. Seeing these kinds of assessment data for ELLS with disabilities we wondered whether and how these students are being provided access to the grade-level standards-based curriculum that precedes the tests. That thought led to our grant project. This comes from the following NCEO report Liu, K., Thurlow, M., Barrera, M., Guven, K. & Shyyan, V. (2005). Graduation exam participation and performance (2000-2001) of English language learners with disabilities (ELLs with Disabilities Report 3). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/ELLsDisReport3.html

7. NCLBInstructional Strategies Teachers must use “scientifically based instructional strategies” to teach challenging academic content to all students. The law stops short of defining what these strategies are. “Schoolwide reform strategies that— (i) provide opportunities for all children to meet the State’s proficient and advanced levels of student academic achievement described in section 1111(b)(1)(D); ‘‘(ii) use effective methods and instructional strategies that are based on scientifically based research that— ‘‘(I) strengthen the core academic program in the school; ‘‘(II) increase the amount and quality of learning time, such as providing an extended school year and before- and after-school and summer programs and opportunities, and help provide an enriched and accelerated curriculum; and ‘‘(III) include strategies for meeting the educational needs of historically underserved populations; ‘‘(iii)(I) include strategies to address the needs of all children in the school, but particularly the needs of low-achieving children and those at risk of not meeting the State student academic achievement standards who are members of the target population of any program that is included in the schoolwide program, which may include— ‘‘(aa) counseling, pupil services, and mentoring services; ‘‘(bb) college and career awareness and preparation, such as college and career guidance, personal finance education, and innovative teaching methods, which may include applied learning and team-teaching strategies; and ‘‘(cc) the integration of vocational and technical education programs; and….”The law stops short of defining what these strategies are. “Schoolwide reform strategies that— (i) provide opportunities for all children to meet the State’s proficient and advanced levels of student academic achievement described in section 1111(b)(1)(D); ‘‘(ii) use effective methods and instructional strategies that are based on scientifically based research that— ‘‘(I) strengthen the core academic program in the school; ‘‘(II) increase the amount and quality of learning time, such as providing an extended school year and before- and after-school and summer programs and opportunities, and help provide an enriched and accelerated curriculum; and ‘‘(III) include strategies for meeting the educational needs of historically underserved populations; ‘‘(iii)(I) include strategies to address the needs of all children in the school, but particularly the needs of low-achieving children and those at risk of not meeting the State student academic achievement standards who are members of the target population of any program that is included in the schoolwide program, which may include— ‘‘(aa) counseling, pupil services, and mentoring services; ‘‘(bb) college and career awareness and preparation, such as college and career guidance, personal finance education, and innovative teaching methods, which may include applied learning and team-teaching strategies; and ‘‘(cc) the integration of vocational and technical education programs; and….”

8. LEP/IEP Strategies Project PROJECT PURPOSE To provide research-based knowledge to educators on the topic of instructional strategies that help middle school ELLs with disabilities achieve in standards-based content classrooms In 2003 we came up with the idea to study this topic in a grant project called “Building & Verifying Effective Instruction Through Consensus for ELLs with Disabilities”. In short hand we call it the LEPIEP Strategies Project. It was funded by OSEP in the fall of 2004 and goes through 2007. We hope to get a one year no-cost extension to finish up some of our activities. Specifically, this project takes a look at instructional strategy use at the school level and the knowledge that teachers possess, placed within the context of specific state standards. We wanted to go to the source, to educators and educational administrators who were being successful instructing ELLs in general (given small numbers of ELLs with disabilities in any one building) to find out what strategies they were using that might not be shared outside of an individual school or district. In designing our project activities we consulted the primary research source on this topic: Gersten, R., Baker, S., and Marks, S. (1998). Teaching English language learners with learning difficulties: Guiding principles and examples from research-based practice. ERIC Document 427 448. We also consulted other relevant articles when designing our project: Artiles, A., & Ortiz, A. (2002). English language learners with special education needs: Contexts and possibilities. In A. Artiles & A. Ortiz (Eds.), English language learners with special education needs: Identification, assessment and instruction (pp. 3–30). McHenry, IL: Center for Applied Linguistics and Delta Systems Co., Inc. Cloud, N. (2002). Culturally and linguistically responsive instructional planning. In A. Artiles & A. Ortiz. (Eds.), English language learners with special education needs: Identification, assessment and instruction (pp. 107–132). McHenry, IL: Center for Applied Linguistics and Delta Systems Co., Inc. Gersten, R., & Baker, S. (2000). What we know about effective instructional practices for English learners. Exceptional Children, 66, 454–470. Müller, E., & Markowitz, J. (2004, March). "Synthesis brief: English language learners with disabilities" project forum at NASDSE. Product PFR-0401. Retrieved March 25, 2005, from: http://www.nasdse.org/publications.cfm In 2003 we came up with the idea to study this topic in a grant project called “Building & Verifying Effective Instruction Through Consensus for ELLs with Disabilities”. In short hand we call it the LEPIEP Strategies Project. It was funded by OSEP in the fall of 2004 and goes through 2007. We hope to get a one year no-cost extension to finish up some of our activities. Specifically, this project takes a look at instructional strategy use at the school level and the knowledge that teachers possess, placed within the context of specific state standards. We wanted to go to the source, to educators and educational administrators who were being successful instructing ELLs in general (given small numbers of ELLs with disabilities in any one building) to find out what strategies they were using that might not be shared outside of an individual school or district. In designing our project activities we consulted the primary research source on this topic: Gersten, R., Baker, S., and Marks, S. (1998). Teaching English language learners with learning difficulties: Guiding principles and examples from research-based practice. ERIC Document 427 448. We also consulted other relevant articles when designing our project: Artiles, A., & Ortiz, A. (2002). English language learners with special education needs: Contexts and possibilities. In A. Artiles & A. Ortiz (Eds.), English language learners with special education needs: Identification, assessment and instruction (pp. 3–30). McHenry, IL: Center for Applied Linguistics and Delta Systems Co., Inc. Cloud, N. (2002). Culturally and linguistically responsive instructional planning. In A. Artiles & A. Ortiz. (Eds.), English language learners with special education needs: Identification, assessment and instruction (pp. 107–132). McHenry, IL: Center for Applied Linguistics and Delta Systems Co., Inc. Gersten, R., & Baker, S. (2000). What we know about effective instructional practices for English learners. Exceptional Children, 66, 454–470. Müller, E., & Markowitz, J. (2004, March). "Synthesis brief: English language learners with disabilities" project forum at NASDSE. Product PFR-0401. Retrieved March 25, 2005, from: http://www.nasdse.org/publications.cfm

9. The conceptual framework for the project is based on the Instructional Framework model developed by AEA 267 in Iowa as a way of thinking about school improvement. Our adaptation of the framework reflects the relationship between improved educational outcomes for students with disabilities, state policy, instructional practice and standards-based assessment. All of the pieces of the conceptual framework fit together like pieces of a puzzle. This emphasizes that increased outcomes for ELLs with disabilities do not happen unless all of the elements are in place. One piece alone, such as standards-based content, is not sufficient. It is the combination of clear state policies on translating the standards to classroom practice for ELLs with disabilities, opportunities to learn standards-based content using appropriate instructional strategies the meet students’ language learning and disability-related needs, and participation in standards-based assessments that creates the increased outcomes for these students.The conceptual framework for the project is based on the Instructional Framework model developed by AEA 267 in Iowa as a way of thinking about school improvement. Our adaptation of the framework reflects the relationship between improved educational outcomes for students with disabilities, state policy, instructional practice and standards-based assessment. All of the pieces of the conceptual framework fit together like pieces of a puzzle. This emphasizes that increased outcomes for ELLs with disabilities do not happen unless all of the elements are in place. One piece alone, such as standards-based content, is not sufficient. It is the combination of clear state policies on translating the standards to classroom practice for ELLs with disabilities, opportunities to learn standards-based content using appropriate instructional strategies the meet students’ language learning and disability-related needs, and participation in standards-based assessments that creates the increased outcomes for these students.

10. Study Phases Phase I: Review of instructional strategies described in state standards and supporting documents Phase II: (1) Educator consensus-building to develop lists of recommended strategies; (2) Validation of recommended strategies via Delphi review; (3) interviews & online surveys with principals Phase III: Dissemination Phase 1 was completed in 2006 and there’s a document on our website with all the details of this study. Look for ELLs with Disabilities Report 18. The on-site components of Phase II were completed. We traveled to all the corners of the U.S. We were in small rural communities, mid-sized towns with poultry processing plants and factories that employed adult immigrants and refugees. We went to one major metropolitan area and one suburban school outside a metropolitan area, although, in general, schools in smaller towns were the ones making AYP in our study states. It was wonderful to interact with educators and principals directly and see their schools in action. We have yet to do the electronic components of the study because we invested a great deal of time and effort into meeting with school staff on site and speaking with them directly. These electronic components (Delphi & survey for principals) are developed and ready to be implemented. We hope to extend the project into 2007-2008.Phase 1 was completed in 2006 and there’s a document on our website with all the details of this study. Look for ELLs with Disabilities Report 18. The on-site components of Phase II were completed. We traveled to all the corners of the U.S. We were in small rural communities, mid-sized towns with poultry processing plants and factories that employed adult immigrants and refugees. We went to one major metropolitan area and one suburban school outside a metropolitan area, although, in general, schools in smaller towns were the ones making AYP in our study states. It was wonderful to interact with educators and principals directly and see their schools in action. We have yet to do the electronic components of the study because we invested a great deal of time and effort into meeting with school staff on site and speaking with them directly. These electronic components (Delphi & survey for principals) are developed and ready to be implemented. We hope to extend the project into 2007-2008.

11. Phase I: 50 State Standards Review RESEARCH QUESTIONS Do states recommend instructional strategies in reading, mathematics, or science for ELLs with disabilities? What is the nature of the instructional strategies being recommended in general, and for ELLs with disabilities? How are the strategies communicated to educators? Many people we’ve talked to have said “Of course standards don’t contain references to instructional strategies” But actually some do. Prior to writing our grant we had scanned some state standards and found one particular state (NJ) that, at that time, had some fairly detailed information on strategies for ELLs in their standards & supporting documents. This made us curious as to what else might be out there and whether there was any information or guidance on strategies that addressed the unique needs of ELLs with disabilities.Many people we’ve talked to have said “Of course standards don’t contain references to instructional strategies” But actually some do. Prior to writing our grant we had scanned some state standards and found one particular state (NJ) that, at that time, had some fairly detailed information on strategies for ELLs in their standards & supporting documents. This made us curious as to what else might be out there and whether there was any information or guidance on strategies that addressed the unique needs of ELLs with disabilities.

12. Definition of Strategy A set of systematic activities used by a teacher that contains explicit steps to achieve a specific student outcome. This set of steps must be replicable by another individual in order to be considered a strategy. This is NOT the only definition of an instructional strategy that exists. We did a scan of research literature and places like What Works Clearinghouse and found many definitions. We created this one because with the NCLB emphasis on proven research-based strategies (we interpreted this to mean those supported with empirical research as to their effectiveness) we had to focus on strategies with specific steps that could be studied and that were explicit enough to be implemented in a variety of settings by different people. Since we were looking at access to the grade-level standards-based content curriculum and the link to outcomes on standards-based achievement tests we decided to focus on strategies used by the teacher to achieve learning outcomes. The next slide details some rules we made for our project about what we would not count as a strategy for our purposes. Here are some words from our report explaining how this definition impacted our findings: Because the definition of an instructional strategy had been shaped to exclude strategies that are taught for their own sake, some of the metacognitive strategies we found would fit that profile, but not all. If a strategy was being employed to target content it was included. We used this narrowed definition because educational literature supports a very broad interpretation of what constitutes a strategy, and the project required a specific focus. This is NOT the only definition of an instructional strategy that exists. We did a scan of research literature and places like What Works Clearinghouse and found many definitions. We created this one because with the NCLB emphasis on proven research-based strategies (we interpreted this to mean those supported with empirical research as to their effectiveness) we had to focus on strategies with specific steps that could be studied and that were explicit enough to be implemented in a variety of settings by different people. Since we were looking at access to the grade-level standards-based content curriculum and the link to outcomes on standards-based achievement tests we decided to focus on strategies used by the teacher to achieve learning outcomes. The next slide details some rules we made for our project about what we would not count as a strategy for our purposes. Here are some words from our report explaining how this definition impacted our findings: Because the definition of an instructional strategy had been shaped to exclude strategies that are taught for their own sake, some of the metacognitive strategies we found would fit that profile, but not all. If a strategy was being employed to target content it was included. We used this narrowed definition because educational literature supports a very broad interpretation of what constitutes a strategy, and the project required a specific focus.

13. For further clarification In this study a strategy is not: A student generated strategy that requires no instruction A student learning strategy acquired through instruction An approach (i.e., a combination of teaching/learning strategies) An assessment activity used to determine placement in or progress through curriculum (e.g., curriculum based measurement) A principle of good teaching (e.g., planning activities before instruction, during instruction, after instruction) Spur of the moment; spontaneous activities suggested by a "teachable moment"(Barrera & Liu, 2005) We provided some more background on this in a strategy definition paper we’re currently working on for publication. In that paper we said: Strategies may be either totally teacher directed, teacher mediated/supported, or used only by the student. Within these 3 types, specific instructional strategies can address 3 main areas: 1.) strategies for acquiring academic content, which contains both content skill development and concept/rule development; 2) strategies for developing effective study skills. and 3) strategies for meeting social/behavioral goals For the purposes of this grant we are only focusing on strategies used by teachers to promote learners’ standards-based academic achievement “ Student Learning Strategies: Student learning strategies acquired through instruction are a large component of pedagogy directed toward remediating the learning difficulties of students with learning disabilities. A rich body of literature exists on these strategies (cf.,Deshler & Lenz, 1989) including in the instruction of English Language Learners (e.g., Ortiz & Wilkinson, 1991; Salend ?and/ or Correa , 19??-find cites). These strategies represent a set of deliberately chosen, systematic activities used by a learner to build individual self-directed learning. In contrast, teaching strategies represent ways that an instructor systematically guides learner acquisition content or skill through the teacher’s pedagogical behavior in a classroom. We provided some more background on this in a strategy definition paper we’re currently working on for publication. In that paper we said: Strategies may be either totally teacher directed, teacher mediated/supported, or used only by the student. Within these 3 types, specific instructional strategies can address 3 main areas: 1.) strategies for acquiring academic content, which contains both content skill development and concept/rule development; 2) strategies for developing effective study skills. and 3) strategies for meeting social/behavioral goals For the purposes of this grant we are only focusing on strategies used by teachers to promote learners’ standards-based academic achievement “ Student Learning Strategies: Student learning strategies acquired through instruction are a large component of pedagogy directed toward remediating the learning difficulties of students with learning disabilities. A rich body of literature exists on these strategies (cf.,Deshler & Lenz, 1989) including in the instruction of English Language Learners (e.g., Ortiz & Wilkinson, 1991; Salend ?and/ or Correa , 19??-find cites). These strategies represent a set of deliberately chosen, systematic activities used by a learner to build individual self-directed learning. In contrast, teaching strategies represent ways that an instructor systematically guides learner acquisition content or skill through the teacher’s pedagogical behavior in a classroom.

14. Process Collected standards and supplementary instructional documents on Internet (e.g., frameworks, teaching resources) Verified accuracy and completeness with state directors Coded documents & analyzed results NOTE: Because of the large amount of documents collected across states we decided to focus on those 10 states with the largest ELL K-12 population at the time, based on statistics from the National Clearinghouse for English Language Acquisition, and the 10 states with the lowest ELL population (N=20). We found more information in supporting documents like curriculum frameworks than in state standards, in general, so we focused on supporting documents except for cases where a state had no supporting documents. Then we looked just at the standards. 19 of 20 state education directors responded with verification of what we’d found. Eight more documents were e-mailed to us during this process. Things we had missed on the Internet.NOTE: Because of the large amount of documents collected across states we decided to focus on those 10 states with the largest ELL K-12 population at the time, based on statistics from the National Clearinghouse for English Language Acquisition, and the 10 states with the lowest ELL population (N=20). We found more information in supporting documents like curriculum frameworks than in state standards, in general, so we focused on supporting documents except for cases where a state had no supporting documents. Then we looked just at the standards. 19 of 20 state education directors responded with verification of what we’d found. Eight more documents were e-mailed to us during this process. Things we had missed on the Internet.

15. Phase I Findings 69 total strategies; 48 directly connected to a standard 1 strategy for ELLs with disabilities (neurological impress method) 11 strategies for ELLs 36 strategies for students with disabilities 21 strategies for general education students One strategy for ELLs with disabilities – neurological impress method/assisted reading mentioned in IL “Using this technique, the teacher sits slightly behind the student and reads along with her/him for not more than 10 to 15 minutes (Purcell-Gates, 1996). During this reading, the teacher needs to maintain a reasonable, fluent pace of reading while the student matches the teacher’s fluency. The teacher runs a finger smoothly along the print as it is being read and the teacher does not use this reading as a basis for working on other aspects such as comprehension or word knowledge.” (Lopez-Reyna, n.d., p.6) Across content areas, many of the strategies recommended for English language learners focused on vocabulary related skills and knowledge, predicting while reading, addressing the concept of similarities and differences, and doing semantic feature analysis. Strategies most often recommended for students with disabilities tended to focus on comprehension, read aloud error rates, addressing the concept of tens and ones and finding the main idea. For general education students, there was no tendency toward any specific skills or knowledge area, though for math, strategies from one state all focused on compounding interest, including several ways to calculate interest (e.g., simple and compound), and angle measures. Overall, recommended strategies across all student groups often involved organizing information through graphical means. MORE RESEARCH IS NEEDED ACROSS A RANGE OF LANGUAGE & CULTURAL BACKGROUNDS AND WITH STUDENTS WITH DIVERSE TYPES OF DISABILITIES.One strategy for ELLs with disabilities – neurological impress method/assisted reading mentioned in IL “Using this technique, the teacher sits slightly behind the student and reads along with her/him for not more than 10 to 15 minutes (Purcell-Gates, 1996). During this reading, the teacher needs to maintain a reasonable, fluent pace of reading while the student matches the teacher’s fluency. The teacher runs a finger smoothly along the print as it is being read and the teacher does not use this reading as a basis for working on other aspects such as comprehension or word knowledge.” (Lopez-Reyna, n.d., p.6) Across content areas, many of the strategies recommended for English language learners focused on vocabulary related skills and knowledge, predicting while reading, addressing the concept of similarities and differences, and doing semantic feature analysis. Strategies most often recommended for students with disabilities tended to focus on comprehension, read aloud error rates, addressing the concept of tens and ones and finding the main idea. For general education students, there was no tendency toward any specific skills or knowledge area, though for math, strategies from one state all focused on compounding interest, including several ways to calculate interest (e.g., simple and compound), and angle measures. Overall, recommended strategies across all student groups often involved organizing information through graphical means. MORE RESEARCH IS NEEDED ACROSS A RANGE OF LANGUAGE & CULTURAL BACKGROUNDS AND WITH STUDENTS WITH DIVERSE TYPES OF DISABILITIES.

16. Findings cont. Limited research base on ELLs with disabilities reflected in lack of references in state documents Existing studies and articles on topic not cited Most strategies cited related to reading strategies for students with disabilities Some strategy descriptions varied across states (e.g., KWL) The KWL examples from several states are cited in ELLs with disabilities report 18, available on our website.The KWL examples from several states are cited in ELLs with disabilities report 18, available on our website.

17. Phase 2– Teacher Consensus Building Research Question In schools throughout the U.S. that are making greater than average progress with English language learners, what instructional strategies do teachers recommend for improving the academic achievement of middle and junior high school English language learners with disabilities in standards-based content instruction? We defined “greater than average progress” as those schools making AYP with a population that included ELLs. In order to make AYP as a school you have to make it with all subgroups including ELLs. So we took 5 top and 5 bottom ELL states and looked for the schools making AYP with ELLS—ELLs with disabilities are typically a very small group in any one building at any one time unless the school has some kind of special program. We thought that if we took schools who were successful with ELLs and students with disabilities then they would also have been successful with ELLs with disabilities and could talk to us about instructional strategies they have used.We defined “greater than average progress” as those schools making AYP with a population that included ELLs. In order to make AYP as a school you have to make it with all subgroups including ELLs. So we took 5 top and 5 bottom ELL states and looked for the schools making AYP with ELLS—ELLs with disabilities are typically a very small group in any one building at any one time unless the school has some kind of special program. We thought that if we took schools who were successful with ELLs and students with disabilities then they would also have been successful with ELLs with disabilities and could talk to us about instructional strategies they have used.

18. Multi-Attribute Consensus Building (MACB) MACB is a quantitative, objective approach for determining a small group’s opinion about the importance of each item on a list . Each person assigned a weighting to each strategy that was brainstormed by the group (we gave them a few strategies from Gersten et al for starters) and we averaged weightings across members of the group. Everyone could then see the average weightings of all the strategies and could discuss differences of opinion and change their own weightings. At least one strategy had to be weighted at 100.MACB is a quantitative, objective approach for determining a small group’s opinion about the importance of each item on a list . Each person assigned a weighting to each strategy that was brainstormed by the group (we gave them a few strategies from Gersten et al for starters) and we averaged weightings across members of the group. Everyone could then see the average weightings of all the strategies and could discuss differences of opinion and change their own weightings. At least one strategy had to be weighted at 100.

19. Participants (n=67) The study sample included 67 educators from ten highly-achieving schools in ten states around the country (35 educators (6 focus groups) participated in our sessions in five states with higher ELL student populations and 32 educators (5 focus groups) joined our sessions in five states with lower ELL student populations). MACB works best with a diverse group and we really wanted to focus on students who crossed service delivery categories so we asked schools for the opportunity to work with all teachers who could comment on standards-based instruction for this group of students. The study sample included 67 educators from ten highly-achieving schools in ten states around the country (35 educators (6 focus groups) participated in our sessions in five states with higher ELL student populations and 32 educators (5 focus groups) joined our sessions in five states with lower ELL student populations). MACB works best with a diverse group and we really wanted to focus on students who crossed service delivery categories so we asked schools for the opportunity to work with all teachers who could comment on standards-based instruction for this group of students.

20. 10 Schools Making AYP We’re using an additional 10 states & schools (5 large, 5 small ELL population) when we do the electronic studies (Delphi, survey) West: 3 schools – 2 with larger ELL populations, 1 with smaller Midwest: 2 schools – 1 with larger, 1 with smaller Northeast: 1 school – 1 with smaller South: 4 schools – 2 with larger, 2 with smaller STRATIFIED RANDOM SAMPLING!! We’re using an additional 10 states & schools (5 large, 5 small ELL population) when we do the electronic studies (Delphi, survey) West: 3 schools – 2 with larger ELL populations, 1 with smaller Midwest: 2 schools – 1 with larger, 1 with smaller Northeast: 1 school – 1 with smaller South: 4 schools – 2 with larger, 2 with smaller STRATIFIED RANDOM SAMPLING!!

21. Phase II: Importance of Content Areas

22. Reading Strategies Chunking and questioning aloud (11) Relating reading to student experiences (11) Using visuals (5) Teacher modeling (3) Checking background knowledge (2) Choral reading (2) Literature circles (2) Multiple reading (2) Predictions (2) Pre-reading survey of text (2) Vocabulary building (2) See handout for list of whole twenty—sometimes differences in weighting were relatively small. Also a page of definitions for these things. We tried to stick as closely as possible to the way the teachers defined things but sometimes we had to supplement the definitions by using research literature. (Frequency across states) First two strategies were given as samples. See handout for list of whole twenty—sometimes differences in weighting were relatively small. Also a page of definitions for these things. We tried to stick as closely as possible to the way the teachers defined things but sometimes we had to supplement the definitions by using research literature. (Frequency across states) First two strategies were given as samples.

23. Curriculum-based Probe Proponent “I think it’s very important that you need to know where the child is at in order to keep going. You cannot keep going unless you know where the child is. So you have to constantly assess, even if it’s informal, it works.” Opponent “I remember doing these as a child, kind of being tested like this in such a quick time frame. And then you know, it even says ‘reach frustration level.’ And I, I just… It frustrates me and it hurts me, and it makes me sad to just think that you have to test them to the point of frustration that such a time limit. Imagine being ELL and Special Ed at the same time trying to do this.” We put this strategy in as one of the two or three initial starter reading strategies from the literature to get people talking. There was more variation in weightings on this one reading strategy than any other strategy and a lot of debate and discussion Curriculum-based probe overall importance – 75.8 (Important) BUT SD = 21.4. SpEd teachers and educators with over 10 yrs of experience considered it “very important”; ESL teachers tended to consider it much less important In the quote from the opponent the speaker is reacting to a definition of a curriculum based probe that was used. It involved repeated timed tasks until student reaches a point where he or she gets to a certain difficulty (frustration) level. This speaker appears to react to the word “frustration”, which is used in certain educational circles but not in the ESL circle! Definition “Having students read aloud three basal reader passages for 1 minute; teacher marks the place where the student stops and then asks comprehension questions and continues to give probes until students reach frustration level as defined by reading rate and median score.” CBM didn’t come up in other content areas.We put this strategy in as one of the two or three initial starter reading strategies from the literature to get people talking. There was more variation in weightings on this one reading strategy than any other strategy and a lot of debate and discussion Curriculum-based probe overall importance – 75.8 (Important) BUT SD = 21.4. SpEd teachers and educators with over 10 yrs of experience considered it “very important”; ESL teachers tended to consider it much less important In the quote from the opponent the speaker is reacting to a definition of a curriculum based probe that was used. It involved repeated timed tasks until student reaches a point where he or she gets to a certain difficulty (frustration) level. This speaker appears to react to the word “frustration”, which is used in certain educational circles but not in the ESL circle! Definition “Having students read aloud three basal reader passages for 1 minute; teacher marks the place where the student stops and then asks comprehension questions and continues to give probes until students reach frustration level as defined by reading rate and median score.” CBM didn’t come up in other content areas.

24. Math Strategies Daily re-looping of previously learned material (11) A student-developed glossary (11) Teacher think-alouds (11) Using manipulatives (8) Relating mathematics to real life (7) Using visuals (4) Drill and practice (2) Hands-on participation (2) Teacher modeling (2) Simplifying problems (2) (Frequency across states) First three strategies were given as samples. (Frequency across states) First three strategies were given as samples.

25. Science Strategies Modeling/teacher demonstration (11) Using pre-reading strategies in science (11) Using pictures to demonstrate steps (11) Hands-on participation (6) Graphic organizers (4) Student-made models (4) Vocabulary development (3) Personal interest research (2) (Frequency across states) First three strategies were given as samples. (Frequency across states) First three strategies were given as samples.

26. Phase II findings No common understanding of what a strategy is. In general teachers were neutral or positive about all strategies. Use of the native language was not mentioned frequently Teachers tended to weight what they used frequently. For reading, these were pre-, during, and post-reading strategies; fluency building; and direct teaching of vocabulary through listening, seeing, reading and writing in short time segments. For mathematics these were tactile, concrete experiences of mathematics, daily re-looping of previously learned material, and problem solving instruction and task analysis strategies. For science, these were hands-on active participation, using visuals, and using pictures to demonstrate steps. It may be that future strategy research using experimental methods will not only help teachers choose the best strategies to fit the population of students in their classrooms, but also to fine tune the implementation of strategies that will work best for students with limited English proficiency, disabilities, or both. Teachers tended to weight what they used frequently. For reading, these were pre-, during, and post-reading strategies; fluency building; and direct teaching of vocabulary through listening, seeing, reading and writing in short time segments. For mathematics these were tactile, concrete experiences of mathematics, daily re-looping of previously learned material, and problem solving instruction and task analysis strategies. For science, these were hands-on active participation, using visuals, and using pictures to demonstrate steps. It may be that future strategy research using experimental methods will not only help teachers choose the best strategies to fit the population of students in their classrooms, but also to fine tune the implementation of strategies that will work best for students with limited English proficiency, disabilities, or both.

27. The top three strategies varied little across types of teachers. Curriculum-based probes or Curriculum based measurement was the most variable  

28. Phase III: Principal Interviews Research Question How and to what degree are state standards that specify instructional strategies translated into practice by educational leaders at the school level?

29. Procedure Semi-structured interviews with 10 principals 5 in high ELL states 5 in low ELL states stratified random sample Schools that made AYP in 2003-2004 with ELLs Four geographic areas Same schools where teacher groups took place What do we say about online principal survey?? Hope to do next year in 07-08 but need no cost extension first.What do we say about online principal survey?? Hope to do next year in 07-08 but need no cost extension first.

30. Interview Questions Questions covered the following issues for ELLs with disabilities: Instructional issues Teacher use of strategies contained in state standards & other documents Sources of teacher information on instructional strategies School or district-provided information on strategies School needs related to instructional strategies

31. Instructional Issues Listed by Principals When we asked principals to name instructional issues they perceived as relevant to teaching ELLs with disabilities these were some of the things they named. Often it was hard to tell whether these issues related to the broader population of ELLs or specifically to the kids we were interested in. Staffing includes high rates of teacher turnover, lack of bilingual staff, lack of ESL teachers Pink– characteristics of students that impact instruction Orange= School-level issues Turquoise = service delivery When we asked principals to name instructional issues they perceived as relevant to teaching ELLs with disabilities these were some of the things they named. Often it was hard to tell whether these issues related to the broader population of ELLs or specifically to the kids we were interested in. Staffing includes high rates of teacher turnover, lack of bilingual staff, lack of ESL teachers Pink– characteristics of students that impact instruction Orange= School-level issues Turquoise = service delivery

32. Many principals believe state standards and supporting documents provide no information on instructional strategies for this population “State documents and strategies are not connected” We’re still writing a report on the principal interview data. We’ve pulled out some themes for you here of things the principals highlighted during interviews. Many of our 10 principals said there was no connection between state standards or supporting documents made available by the state and instructional strategies. We’re still writing a report on the principal interview data. We’ve pulled out some themes for you here of things the principals highlighted during interviews. Many of our 10 principals said there was no connection between state standards or supporting documents made available by the state and instructional strategies.

33. What do principals and teachers rely on for instructional strategy information? If that’s true, then we wanted to know what resources staff were using to make decisions about how best to teach ELLs with disabilities in mainstream classrooms.If that’s true, then we wanted to know what resources staff were using to make decisions about how best to teach ELLs with disabilities in mainstream classrooms.

34. Teaching models with packaged materials “Our school has just adopted the SIOP Model for teaching ESL kids….It has all the strategies for reading and writing and they are research based. Our ESL teachers use them for lesson planning and designing many other class activities…We have the SIOP Model book…” Here’s an example from a school along the U.S.-Mexico border that had a high percentage of ELLs in the building. In this particular school the SIOP model, which has been described by some as a research-based approach to sheltered instruction (using strategies to teach content in comprehensible ways to ELLs), appears to have been used only with students in ESL classrooms. ELLs with disabilities may not always receive service in ESL classrooms if they are in special education so it’s unclear to what extent ELLs with disabilities were taught with the strategies contained in these materials. However, this school had a high percentage of second language learners in its student body so the SIOP model may have had fairly wide implementation. The principal did not mention any specific examples of strategies contained in these materials. Instead he referred to the knowledge base of the ESL teachers and to support from the district. It sounded as if he was not directly involved in instructional strategy decision-making for his diverse student body. Echevarria, J., Vogt, M. E., & Short, D. (2004). Making content comprehensible for English learners: The SIOP model. Boston: Allyn & Bacon.Here’s an example from a school along the U.S.-Mexico border that had a high percentage of ELLs in the building. In this particular school the SIOP model, which has been described by some as a research-based approach to sheltered instruction (using strategies to teach content in comprehensible ways to ELLs), appears to have been used only with students in ESL classrooms. ELLs with disabilities may not always receive service in ESL classrooms if they are in special education so it’s unclear to what extent ELLs with disabilities were taught with the strategies contained in these materials. However, this school had a high percentage of second language learners in its student body so the SIOP model may have had fairly wide implementation. The principal did not mention any specific examples of strategies contained in these materials. Instead he referred to the knowledge base of the ESL teachers and to support from the district. It sounded as if he was not directly involved in instructional strategy decision-making for his diverse student body. Echevarria, J., Vogt, M. E., & Short, D. (2004). Making content comprehensible for English learners: The SIOP model. Boston: Allyn & Bacon.

35. “…We take an interest in differentiated instruction and that really takes on the aspect of all students and different types of learning…one of the strategies is to make a menu-type of assignment where they will have like an appetizer, and they will have a main course and a dessert and under each of those categories there are different assignments…The ESL students have been doing things like bringing in food and clothing… and that could be a part of their assignment” Here’s another school that mentions something that may be packaged materials. ASCD is a big promoter of differentiated instruction. They offer packaged sets of materials for staff development, largely written by Carol Ann Tomlinson, videos, etc. The resource guides come with units already developed and include differentiation strategies from teachers who have taught the units. It’s unclear how this school is incorporating differentiated instruction and whether they have access to these materials or what specific strategies from the materials they might be using. However, if the ESL kids in general are involved in content instruction only through peripheral activities like bringing in food and clothing to represent their home culture, then access to the grade level content for ELLs with disabilities is certainly in question. This principal did not mention having been a teacher when we asked him. He was relatively new to being a principal and worked in a school in a rural area that had very limited access to ESL teachers. His school did not have a full time ESL teacher and he talked about the lack of financial resources to provide what ELLs needed. This particular school was in a state with low numbers of ELLs in general but the rural school had experienced an influx of students due to a nearby factory that employed workers from other countries. The school had very limited resources for teaching ELLs and had only 1/5 of an itinerant ESL teacher.Here’s another school that mentions something that may be packaged materials. ASCD is a big promoter of differentiated instruction. They offer packaged sets of materials for staff development, largely written by Carol Ann Tomlinson, videos, etc. The resource guides come with units already developed and include differentiation strategies from teachers who have taught the units. It’s unclear how this school is incorporating differentiated instruction and whether they have access to these materials or what specific strategies from the materials they might be using. However, if the ESL kids in general are involved in content instruction only through peripheral activities like bringing in food and clothing to represent their home culture, then access to the grade level content for ELLs with disabilities is certainly in question. This principal did not mention having been a teacher when we asked him. He was relatively new to being a principal and worked in a school in a rural area that had very limited access to ESL teachers. His school did not have a full time ESL teacher and he talked about the lack of financial resources to provide what ELLs needed. This particular school was in a state with low numbers of ELLs in general but the rural school had experienced an influx of students due to a nearby factory that employed workers from other countries. The school had very limited resources for teaching ELLs and had only 1/5 of an itinerant ESL teacher.

36. Locally developed curriculum maps “We look at the state standards and those are our…we have curriculum maps that are taught around those…The curriculum maps were then developed based upon the state standards, so the teachers…it’s been dictated, it’s not a choice, you have to follow these curriculum maps. And the curriculum maps have a pacing scale to them…The teachers are… refer to the curriculum maps, not to state standards.” These curriculum maps were created by the district. Curriculum maps vary a great deal and can contain specific references to instructional strategies or can be very general and describe only topics students will learn and the timing of instructional units. This principal did not mention any specific strategies contained in those curriculum maps.These curriculum maps were created by the district. Curriculum maps vary a great deal and can contain specific references to instructional strategies or can be very general and describe only topics students will learn and the timing of instructional units. This principal did not mention any specific strategies contained in those curriculum maps.

37. ESL or Bilingual Specialists  P1: “ We rely heavily on [name], District Specialist, and she has provided various trainings…She is very thorough, professional, and efficient” P2: “Our ELL teacher as well as our reading teacher, they spent a lot of time training the staff” A few principals talked about delegating a lot of the responsibility for instructional decision making on ELLs with disabilities to other staff. Most often they referred to the ESL teacher or district ESL specialist as that person. Interestingly, no principal mentioned the special ed department as taking the lead in recommending instructional strategies for ELLs with disabilities. The ESL staff seemed to be viewed as the ones who held the relevant knowledge.A few principals talked about delegating a lot of the responsibility for instructional decision making on ELLs with disabilities to other staff. Most often they referred to the ESL teacher or district ESL specialist as that person. Interestingly, no principal mentioned the special ed department as taking the lead in recommending instructional strategies for ELLs with disabilities. The ESL staff seemed to be viewed as the ones who held the relevant knowledge.

38. Locally Provided Professional Development P1: “We had a five year grant in the district for professional development for regular teachers. During that…grant, we did the training with staff across the district” P2: “Teachers are always getting trained, we have professional development programs, money, and incentives, ESL Summer Institute…for the teachers who could not attend it we have a back up program, we have good college connections and good relationships with curriculum people in the district” A couple of principals mentioned specific teacher development opportunities where teachers learned about instructional strategies for students but it was unclear whether they were referring to ESL students as a whole or to ELLs with disabilities. It looks like principal 2 is at least partly speaking of training on ELLs as a whole. These two principals did not mention specific strategies contained in those trainings.A couple of principals mentioned specific teacher development opportunities where teachers learned about instructional strategies for students but it was unclear whether they were referring to ESL students as a whole or to ELLs with disabilities. It looks like principal 2 is at least partly speaking of training on ELLs as a whole. These two principals did not mention specific strategies contained in those trainings.

39. District Resources “Our district provides specific strategies for ELL and special ed students and our teaching staff follow them and adapt them” Note: This comment was made by the principal of the school that used the SIOP model and had teaching materials associated with it. Several principals spoke about the important role of district staff in instructional strategy decision making…either through guiding implementation of a curriculum and/or providing inservice training.Note: This comment was made by the principal of the school that used the SIOP model and had teaching materials associated with it. Several principals spoke about the important role of district staff in instructional strategy decision making…either through guiding implementation of a curriculum and/or providing inservice training.

40. Mixture “We do a lot of strategies. That is just a given with our school. We go through many different areas that have been highlighted, especially in the area of sheltered instruction for our students, scaffolding, we do a lot with techniques that the state sees very important for us to work with…I’ll say that our school here pretty much across the board implements many different strategies to help ELL students out…anywhere from brain-based learning to Renaissance to reading strategies” “ This principal is from a small rural school that happened to have a sizeable amount of money due to the tourism industry. When our research team visited he handed us a locally developed manual on teaching ELLs with disabilities. The school appears to use a variety of things this principal sees as strategies. He first refers to sheltered instruction which is an instructional model for English language learners in general where content is taught in particular ways to ELLs. Some professionals refer to sheltered instruction as a strategy although multiple strategies can be used to make content linguistically appropriate for ELLs. It’s not clear here what strategies the school is using during sheltered instruction. Then the principal goes on to mention techniques the state sees as important (again unspecified—he never answered the question about strategies contained in state documents). Next he mentions Brain Based learning, which is often discussed as a theory of how people learn and how the brain processes information. Some proponents of brain based learning discuss particular instructional techniques that go with this theory of learning. Examples are reducing anxiety in students, allowing for active processing of information, etc. It does not seem to be an approach that has any specific strategies aimed at linguistically and culturally diverse students or students with disabilities. We interpreted Renaissance to mean the Accelerated Reader program put out by Renaissance Learning. What Works Clearinghouse reviewed and apparently approved Accelerated Reader as an intervention. On the WWC it describes the software this way: Accelerated Reader “is a computer program that facilitates reading practice by providing students and teachers feedback from quizzes based on the books students read. The program gives students the opportunity to practice reading books at their level, provides feedback on student comprehension of books and helps students establish goals for their reading.” Reading strategies could be specifically for students with disabilities but that’s also not clear. This principal is from a small rural school that happened to have a sizeable amount of money due to the tourism industry. When our research team visited he handed us a locally developed manual on teaching ELLs with disabilities. The school appears to use a variety of things this principal sees as strategies. He first refers to sheltered instruction which is an instructional model for English language learners in general where content is taught in particular ways to ELLs. Some professionals refer to sheltered instruction as a strategy although multiple strategies can be used to make content linguistically appropriate for ELLs. It’s not clear here what strategies the school is using during sheltered instruction. Then the principal goes on to mention techniques the state sees as important (again unspecified—he never answered the question about strategies contained in state documents). Next he mentions Brain Based learning, which is often discussed as a theory of how people learn and how the brain processes information. Some proponents of brain based learning discuss particular instructional techniques that go with this theory of learning. Examples are reducing anxiety in students, allowing for active processing of information, etc. It does not seem to be an approach that has any specific strategies aimed at linguistically and culturally diverse students or students with disabilities. We interpreted Renaissance to mean the Accelerated Reader program put out by Renaissance Learning. What Works Clearinghouse reviewed and apparently approved Accelerated Reader as an intervention. On the WWC it describes the software this way: Accelerated Reader “is a computer program that facilitates reading practice by providing students and teachers feedback from quizzes based on the books students read. The program gives students the opportunity to practice reading books at their level, provides feedback on student comprehension of books and helps students establish goals for their reading.” Reading strategies could be specifically for students with disabilities but that’s also not clear.

41. Phase III Findings Most principals in our study were relatively uninvolved in directly translating instructional strategies to practice. Much of the strategy guidance came from packaged curricula or professional development programs, from district specialists, or from building specialists. Principals often had knowledge of, or directly supervised the staff member who was doing the translation of strategies to practice. They often had some knowledge of the approach and/or materials used, but may not have known the specific strategies taught. Overall, principals seemed more able to talk about the curricula or the professional development programs than any other aspect of teaching ELLs with disabilities. Often their comments related to preparing staff to work with ELLs as a whole, and not specifically to the needs of ELLs with disabilities. They may see relatively few ELLs with disabilities but given that these were schools making AYP it was surprising to us not to see morePrincipals often had knowledge of, or directly supervised the staff member who was doing the translation of strategies to practice. They often had some knowledge of the approach and/or materials used, but may not have known the specific strategies taught. Overall, principals seemed more able to talk about the curricula or the professional development programs than any other aspect of teaching ELLs with disabilities. Often their comments related to preparing staff to work with ELLs as a whole, and not specifically to the needs of ELLs with disabilities. They may see relatively few ELLs with disabilities but given that these were schools making AYP it was surprising to us not to see more

42. Preliminary Project Findings State documents do not provide a great deal of guidance Teachers generated a list of untargeted strategies Principals often delegate instructional decisions for these students It appears that there really is no “best practices” knowledge base among educators and principals who work with ELLs with disabilities in schools making AYP. One of the original goals of our project was to find out what practices were working and share those but most of what we heard about from school staff targeted the ELL needs of these students, not the SpEd needs, and appeared to be knowledge possessed and handed down by ESL teachers or by district teacher development staff. To us, the findings so far indicate a place where much more research is needed to show what kinds of strategies are most effective for students who have both types of needs. These kids may be one of the elephants in the room—they’re just not being talked about or addressed directly in many schools! Disseminating research information in comprehensible ways for teachers, teacher trainers and principals appears to also be a huge need. Not one principal or teacher referred to research as being a factor in making strategy decisions, even though NCLB requires that research based strategies be used. It appears that there really is no “best practices” knowledge base among educators and principals who work with ELLs with disabilities in schools making AYP. One of the original goals of our project was to find out what practices were working and share those but most of what we heard about from school staff targeted the ELL needs of these students, not the SpEd needs, and appeared to be knowledge possessed and handed down by ESL teachers or by district teacher development staff. To us, the findings so far indicate a place where much more research is needed to show what kinds of strategies are most effective for students who have both types of needs. These kids may be one of the elephants in the room—they’re just not being talked about or addressed directly in many schools! Disseminating research information in comprehensible ways for teachers, teacher trainers and principals appears to also be a huge need. Not one principal or teacher referred to research as being a factor in making strategy decisions, even though NCLB requires that research based strategies be used.

43. Teachers may not have strong building-level leadership on teaching ELLs with disabilities Few information sources teachers rely on address the specific needs of ELLs with disabilities It’s unclear how many of sources of information are research based.