1 / 49

AMSTEL – Seminar Pauline Vos

Universiteit van Amsterdam AMSTEL Instituut. Benefits and limitations of international comparative research in science and mathematics education. AMSTEL – Seminar Pauline Vos. International comparison.

zlata
Download Presentation

AMSTEL – Seminar Pauline Vos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Universiteit van Amsterdam AMSTEL Instituut Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos 22 April 2009

  2. International comparison • Use identical instruments in different countries and look at differences between countries. • My own international experience: • Went to school in NL, D and F • worked in NL, Zimbabwe, Mozambique & South Africa • Worked at the Dutch research centre for TIMSS-1999 (University of Twente) • PhD on mathematics curriculum in NL using TIMSS-instruments

  3. Many variations • Bilateral studies • Affect in science: ROSE (Relevance of Science Education), 40 countries • Teacher competencies: TEDS(M), 17 countries

  4. Two abbreviations… • PISA = Project for International Student Assessmentorganised by OECD(Organisation for Economic Cooperation and Development) • TIMSS = Trends in International Maths and Science Studypreviously: Third Internat. Maths & Science Studyorganised by IEA (Internat. Assoc. for the Evaluation of Educational Achievement)

  5. Translate and define • Mathematics • Basic calculations and further • Science • physics, chemistry, biology, physical geography, astronomy, ecology, etc • Students • all students (not just pre-university)

  6. Some history… • 1964 A comparison of 12 countries, Husén & Bloom • 1971 FISS • 1980-82 SIMS • 1984 SISS • 1995 TIMSS (later known as “TIMSS-1995” ) • 1999 TIMSS-Repeat (later known as “TIMSS-1999”) • 2003 TIMSS-2003 • 2007 TIMSS-2007 • 2008 TIMSS Advanced TIMSS-2011 • 2000 PISA-2000 • 2003 PISA-2003 • 2006 PISA-2006 PISA-2009

  7. How can you do research on the question: To what extent do countries differ in the education of science and mathematics? 22 April 2009

  8. Hours of instruction for science (grd 4)

  9. Teacher characteristics math (grd 8)

  10. Both studies focus on a test 22 April 2009

  11. Complex studies .. • Assessment framework • Large number of items • Long design process for instruments • Large random sample • Quality control • trend items (retained) • coding and re-coding • excluding countries when criteria were not met

  12. Pisa framework for science

  13. Pisa framework for science competencies

  14. Pisa framework for science

  15. Pisa framework for science

  16. Pisa science item ‘Acid Rain’

  17. Pisa science item ‘Acid Rain’

  18. Pisa science item ‘Acid Rain’

  19. Complex studies… (1) • Not just measuring country scores in one discipline, but also: • Sub-disciplines (algebra, ecology, ..) • Sub-competencies (reproduce, explain, communicate, ..) • Background variables • Not just measuring students’ scores, but also trying to explain: • Differences between countries • Differences between schools • Differences between students

  20. Complex studies… (2) • PISA: 85 exercises distributed over 13 test booklets • PISA-2000 • Main topic: Literacy (reading) • Additional: science, maths • PISA-2003: • Main topic: Mathematical Literacy • Additional: literacy, science, problem solving • PISA-2006 • Main topic: Science • Additional: literacy, maths

  21. Complex studies… (3) • TIMSS: 300 exercises distributed over 12 booklets • TIMSS-95: • grades 3, 4, grades 7, 8 • Performance Assessment (= test in a lab-like environment) • grade 12: math/science literacy; advanced maths/science • video study (USA, Jap, Germ.) • TIMSS-99: • grade 8 • TIMSS-R Video study (USA, Jap, NL, Cz, Switz, Austr, HongK.) • TIMSS-2003 • grade 4, grade 8 • TIMSS-2007 • grade 4, grade 8 • TIMSS-2008 • grade 12

  22. Percentage Distribution Math Items PISA - TIMSS

  23. Math Item Formats *PISA definitions: Multiple Choice includes regular and “complex” multiple-choice items. Short Constructed Response includes “short answer” and “closed response” items. Extended Constructed Response includes “open response” items.

  24. Mathematical Complexity

  25. PISA-2006 science country ranking

  26. PISA-2006 science country ranking

  27. TIMSS-2007country rankingMaths grade 4

  28. TIMSS-2007 multi-country comparison

  29. Horse race - thinkingboth in PISA and TIMSS • Ranking is simplistic • Sample: intrinsically inaccurate • Scores of some countries are very close. With s.e.>3 are small differences not significant • NL position in 2003: • math: on shared positions 2-7 in PISA and shared positions 6-9 in TIMSS • science: on shared positions 4-8 in PISA and shared positions 7-11 in TIMSS

  30. Differences PISA and TIMSS • Population: • PISA: age cohort: 15-year olds (indep. of grade) • TIMSS: grade (indep. of age) • Countries: • PISA: economically developed (OECD) • TIMSS: whole world (incl. Saudi Arabia, Botswana) • Cyclus: • PISA: 3-yearly with different thematic foci • TIMSS: 4-yearly with different populations • Content: • PISA: applicable knowledge (situations): reading, math. literacy, scientific literacy • TIMSS: school knowledge (US curriculum): mathematics, science • Justification of the study: • PISA: knowledge has economical importance • TIMSS: knowledge about education is needed

  31. Country score and GDP

  32. Grade 8 seven countries: Australia, Hong Kong, Japan, NL, Czech Rep, USA, Switzerland Aim: describe patterns in lessons question: what is typical for the math lessons in a certain country? TIMSS-1999 Video Study • in each country 85 random lesson filmed, transcribed & coded • Points of analysis: • Structure of the lesson: e.g. who speaks? How many exercises? • Mathematical content: complexity, procedures • Char of exercise: symbols, contexts

  33. TIMSS-Video study: use of materials: • Percentage lessons in which a calculator is used

  34. TIMSS-Video study: structure of lessons • Percentage lessons in which a goal is stated

  35. TIMSS-Video study: structure of lessons • Percentage lessons, in which a summary statement is made

  36. TIMSS-Video study: exercises • Percentage exercises per lesson with a context of symbols only

  37. USA video USA Grade 8 (14 yrs) Heterogeneous non-problem segment:the teacher shows the class examples of angles formed by secants and tangents intersecting inside, on, or outside a circle. TIMSS-1999 Video Study

  38. Netherlands video NL Grade 8 (14 yrs) Homogeneous (pre-university) non-problem segment:the teacher briefly presents historical background related to the Pythagorean theorem (i.e., Pythagoras was a Greek gentleman). She also connects ideas learned previously (i.e., the definitions of various types of triangles) to the present lesson. TIMSS-1999 Video Study

  39. Hong Kong: grd 8, heterogeneous Explanation of a new concept. video HongKong Pre-knowledge: students can solve linear equations What preceded the scene: Solving2x+10=2(x+5)2x+10=2x+10 0 = 0 Then different values were insterted for x . left hand sideright hand side“expanded form”“factorized form” TIMSS-1999 Video Study

  40. Next in this theatre… • TEDS(M) report in December 2009 • TIMSS-2008 Advanced report in December 2009 • Pisa 2009 report in 2010

  41. References for PISA and TIMSS • www.pisa.oecd.org • http://timss.bc.edu

  42. remains the question…. what are the benefits and limitations?

More Related