1 / 19

Assessment in the Less Commonly Taught Languages: Challenges and Approaches

Assessment in the Less Commonly Taught Languages: Challenges and Approaches. William P. Rivers, Ph.D. Executive Director Joint National Committee for Languages National Council for Languages and International Studies September 23, 2013. Overview. Overview Operational Definition of LCTLs

nevaeh
Download Presentation

Assessment in the Less Commonly Taught Languages: Challenges and Approaches

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Assessment in the Less Commonly Taught Languages: Challenges and Approaches William P. Rivers, Ph.D. Executive Director Joint National Committee for Languages National Council for Languages and International Studies September 23, 2013

  2. Overview • Overview • Operational Definition of LCTLs • LCTL vs. “Critical Language” • Common characteristics of LCTL fields • Basic types of language assessment • Achievement • Proficiency • Why FL has an edge in assessment: Proficiency • Challenges in proficiency testing in the LCTLs • Alternative approaches • About JNCL-NCLIS • Q & A

  3. What is a “Less Commonly Taught Language?” • MLA enrollment survey (Furman et al., 2009): • 1.6m FL enrollments in US IHEs in 244 FLs • 8.4 enrollments per 100 students (essentially unchanged since 1974) • 1.2m of these (80%): Spanish, French, German • In effect, all other FLs are LCTLs • 1st tier: 10,000 – 80,000 enrollments: Chinese, Italian, Japanese, Russian, Portuguese, Hebrew, Japanese • 2nd tier: 1,000 – 10,000 enrollments: 9 • 3rd tier: <1,000: 235 FLs

  4. Common characteristics of LCTL fields: Capacity challenges • Foundational: • linguistic descriptions may be lacking; SLA research in the TL may be lacking • Few or no Ph.D. programs to train future faculty & researchers • Few or no programs to train K-12 teachers • Infrastructure: • fewer and less-widely distributed disciplinary journals • Smaller or joint conferences • Few professional development programs • Programs • Fewer programs (of course) and • Programs with no advanced level programming • Programs under constant threat of elimination due to low enrollments

  5. LCTL vs. “Critical Language” • Each USG agency has its own criteria for determining which languages are critical • Not all LCTLs are critical • Not all CTLs aren’t critical • For external funding programs, determination of “critical language” rests on • Agency needs • Programmatic priorities • funding

  6. Basic types of FL Assessment • Achievement: Discrete point testing of specific items • Typically tied to a specific course or curriculum • Proficiency: graduated, structured, progressive testing of language competencies • Curriculum and program independent • Fundamentally communicative • Proficiency testing began at Foreign Service Institute in the 1950s – developed by John Carroll and his students • Based on three years’ observation of language use by diplomats • Divided into 4 skills (L/R/S/W) and five levels • This became the basis for the ILR Proficiency Scale • In turn the basis for the ACTFL proficiency Scale (and most others worldwide use the same approach)

  7. Proficiency Scale Review • ACTFL: • Novice • Intermediate • Advanced • Superior • ILR: 0 -5 • Professional language skill generally requires the “Superior” or ILR 3 level • 4 years of College FL results in a median proficiency of Intermediate High /1 +

  8. Why we have an edge • FL proficiency testing has revolutionized FL pedagogy in the past 30 years: we now focus on communicative ability as the fundamental goal of FL education • Among the Humanities, FL has a unique capability to show what students can do with their knowledge, because proficiency as a construct is precisely that – what can an individual dowith their FL • 50+ years of practice and R&D in FL testing

  9. LCTL Assessment Challenges • Range of languages • 240 + languages • Geographic dispersion of examinees • Low number of examinees • Economic viability of test development • Technical challenges • Validation • Test design

  10. An Incomplete Set of Available Tests: Oral Proficiency Interviews Afrikaans, Akan-Twi, Albanian, Algerian, Amharic, Arabic, Armenian, Azerbaijani, Baluchi, Bengali, Bosnian, Bulgarian, Burmese, Cambodian, Cantonese, Cebuano, Chavacano, Czech, Dari, Dutch, Egyptian, English, French, Ga, Georgian, German, Greek (Modern), Gujarati, Haitian Creole, Hausa, Hebrew, Hiligaynon, Hindi, Hmong-Mong, Hungarian, Igbo, Ilocano, Indonesian, Italian, Iraqi, Japanese, Javanese, Jordanian, Kashmiri, Kazakh, Kikongo-Kongo, Korean, Krio, Kurdish Kurmanji, Kurdish Sorani, Lao, Lebanese, Levantine, Libyan, Lingala, Malay, Malayalam, Mandarin, Mandingo-Bambara, Moro, Nepali, Pashto, Persian-Farsi, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian/Croatian, Sindhi, Sinhalese, Slovak, Somali, Spanish, Swahili, Tagalog, Tajik, Tamil, Tausug, Telugu, Thai, Tigrinya, Turkish, Turkmen, Uighur, Urdu, Uzbek, Vietnamese, Wolof, Wu, Yoruba   Source: Language Testing International

  11. Challenge: volumeApproach: Self-Assessments • 4 skill (L/R/S/W) • Based on ILR scale • Global statements of proficiency • Detailed assessments by task and proficiency level (Can-do statements) • Regular monitoring of concurrent validity worth formal proficiency tests

  12. Self-Assessments: Concurrent Validity • 4 skill (L/R/S/W) • Based on ILR scale • Global statements of proficiency • Detailed assessments by task and proficiency level (Can-do statements) • Regular monitoring of concurrent validity with formal proficiency tests • Peer reviewed (Stansfield, Gao, Rivers, 2010)

  13. Self-Assessments: Concurrent Validity • Moderate correlations; acceptable for membership screening

  14. Challenge: Range of LanguagesApproach: Expand access to quality tests • ASTM F2889 – 11: National industry standard for proficiency test development • Users and providers develop consensus Standard to ASTM format • ASTM: American Society for Testing and Materials • 30000+ industry standards • Develops but does not enforce standard • Equal representation of users & providers • Participation is open & voluntary • ASTM F43 on Language Services and Products • 120 members from industry, academe, government • 4 standards developed • 6 underway

  15. Value of Standard Practice • Leverage • world-leading foreign language proficiency testing (the DLPT system) and practices (the FILR Proficiency Guidelines) to ensure quality procurement of test development and administration • the $15b language services industry’s potential to provide language proficiency testing in more than 150 languages • Enable rapid development of foreign language proficiency tests to meet surge requirements • Enhance compliance with • PL 104-113 §12(d), which requires US Government agencies to maximize the use of industry standards • DoD Standards Program, DoD 4120.24-M • Improve quality assurance through the availability of such standards to procurement and contracting for language proficiency testing

  16. Scope of ASTM F2889-11, Standard Practice for Assessing Language Proficiency • Establishes specific requirements for quality and transparency in • Test specification • Test development • Test administration • Incorporates current best practices • FILR Guidelines • DLPT-V testing • Other USG and industry leading test practices • All levels/all modalities are covered

  17. Scope of Standard Practice (2) • Incorporates current best practices • FILR Guidelines • DLPT-V testing • Other USG and industry leading test practices • Mandates specific practices, e.g., • Test frameworks • Ethics • Quality control • Quality assurance

  18. The Joint National Committee for Languages and the National Committee for Languages and International Studies • Comprise more than 80 organizations in the language enterprise • Develop and advocate policies for language and global expertise • Promote language as a profession in the United States • Support the development of academic, governmental, and industrial standards for language work • Have joined with the language industry and the Globalization and Localization Association to form the American Language Enterprise Advocacy (ALEA)

  19. Contact Dr. Bill RiversExecutive Director, JNCL-NCLIS Chair, ASTM Main Committee F43, Language Services and Products wrivers@languagepolicy.org

More Related