National Assessment of Adult Literacy

1. National Assessmentof Adult Literacy NAAL and Adult English Language Learners Mark Kutner American Institutes for Research 202 944 5321 mkutner@air.org Issues and Challenges in Assessment & Accountability Center for Applied Linguistics May 16, 2003

2. NAAL Definition of Literacy. Literacy is using printed and written information to function in society, to achieve one’s goals, and to develop one’s knowledge and potential. • Nationally representative household survey of at least 20,000 adults providing estimates of (1) English literacy skills of adults 16 years of age and older, (2) changes in literacy levels between 1992 and 2003.

3. NAAL consists of three blocks of items (not multiple choice) measuring prose, document, and quantitative literacy. • Stimulus materials for literacy tasks consist of representations of actual materials used by adults in the United States in their everyday lives. • Advertisements and utility bills • Health materials • City and transportation maps, menus • Newspaper and magazine articles

4. Two New Components • NAAL addresses shortcomings of the analyses conducted for adult ESL learners in 1992 by including two new components: • Fluency Addition to NAAL (FAN). Assesses the ability of adults to decode, recognize words and numbers, and to read with fluency. • Adult Literacy Supplemental Assessment (ALSA). Assesses the ability of low-literate adults to use printed information from actual products and materials.

5. Health Literacy • NAAL will also assess health literacy of ESL adults using HHS definition: The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions. • Three domains of health literacy: clinical information, prevention activities, navigating the health care system.

6. Background Questionnaire Core Cognitive Items ALSA NAAL FAN Flow of Respondents

7. FAN • FAN explicitly assesses basic reading skills to provide meaningful data about lower performing adults, including English language learners. • FAN will measure decoding, word recognition, and fluency skills. • A separate “basic skills score” will be provided, and the relationship between basic skills and NAAL scores will be described. • Instructions and questions will be available in Spanish.

8. FAN • FAN tasks include: • Baseline measure of non-reading speed • Letter lists • Number lists • Word lists • Nonsense word lists • Oral reading passages accompanied by comprehension questions

9. ALSA • A performance-based assessment for adults at the lowest levels of English literacy. • ALSA describes how low-literate adults negotiate English print and what materials they use, even if they cannot read connected text. • ALSA respondents include native English speakers who are low-literate and Spanish speakers of varying levels of literacy whose English is not proficient enough to take the NAAL.

10. ALSA • Oral directions and questions are available in Spanish, and respondents may answer in Spanish. • Approximately 5 percent of NAAL respondents will participate in ALSA. • Respondents are identified in the field, based on their performance on a set of six cognitive items scored by interviewer.

11. ALSA • All stimulus items are authentic. They are actual items adults encounter in everyday life, not representations of the items. The stimulus items represent a continuum of difficulty, and include: Common products • Coca-Cola can • Common food containers (e.g., cracker boxes, bags of rice or beans) • Medicine and health aid labels

12. ALSA Popular Pictures and Printed Material • Advertisements • TV Guide Flyers and Announcements • Store sales • Yard sales Mail Items • Bill or other common piece of mail

13. ALSA • ALSA task types measure: • Letter identification • Number identification • Word reading • Word identification • Comprehension • Familiarity with materials • Use of materials • ALSA is based on AIR’s reading demonstration assessment used in the “What Works” Study for Adult ESL Literacy Students.

14. Literacy Skills Included in ALSA and FAN • Vocabulary knowledge • Fluency • Timed reading • Words read correctly • Strategies • Locating information through skimming and scanning • Literacy practices • Use of texts • Literacy “helpers” or “brokers” • Recognition of sight words, including • Logos and symbols • Numbers • Decoding, alphabet knowledge • Letter names • Sounds and syllables • Meaning Making • Pragmatics (purpose, function) • Comprehension