Introducing Desirable Difficulties for Educational Applications in Science

1. Introducing Desirable Difficulties for Educational Applications in Science www.psych.ucla.edu/iddeas Robert A. Bjork University of California, Los Angeles Marcia C. Linn University of California, Berkeley

2. “Desirable Difficulties” • Spacing rather than massing study • Interleaving rather than blocking practice on separate topics or tasks • Varying contextual cues • Reducing feedback to the learner • Testing rather than re-presenting

3. Learning versus performance • What we can observe is performance, what we must infer is learning… – and the former is an unreliable index of the latter • Instructors are, therefore, susceptible to choosing less-effective conditions of learning over more-effective conditions • And as learners, we, too, are susceptible to confusing performance with learning

4. Generation Interleaving Spacing

5. Goals of the IDDEAS project • Do such findings extend to to-be-learned materials and retention intervals that are realistic from an educational standpoint? • And, more broadly, what design principles are fundamental in optimizing educational materials and practices?

6. WISE (web-based inquiry science environment): http://wise.berkeley.edu • Advantages as a tool for teachers – Supports authoring and customization – Contains a library of tested projects – Enables collaborative learning, visible thinking, autonomous investigation – Transportable • Advantages as an IDDEAS research tool

7. Current Projects WISE Platform • 2 laboratory studies, UCLA • 2 classroom studies, UCB Design Principles • 2 studies, UCLA

9. Interleaving • Motor tasks: patterns, force production, bank machine transactions (Lee & Magill, 1983, Simon & Bjork, 1990 Charles et. al, 1990, Jamieson & Robers, 2000) • Sports: badminton, volleyball, baseball (Bortoli et al, 1992, Goode & Magill, 1986, Hall et al, 1994) • Abstract learning tasks: mazes, tracking (Carleson et.al, 1989, Jelsma & Van Merrienboer, 1989, Jelsma & Pieters, 1989) • Logic rules, boolean operators (Schneider et al, 1995, Carleson & Yaure, 1990)

11. Example of generation effects

12. Study 1: Overview • Uses an existing WISE astronomy module: – How scientists determine the habitability of planets • Design:4 groups – Mass and distance slides either blocked or interleaved; – Some evidence re-studied via generation or reading • Forty-eight hour retention interval • Posttest

13. Posttest questions • Simple sentence-completion: Information presented and re-studied via generation or reading, or only presented – E. g. “The amount of heat and light emitted by the sun in our solar system has increased by ____% since the beginning of earth’s history.” • Integration questions across mass slides or across distance slides – E. g., “Would an object weigh more on the planets in our solar system made mostly of gas or made mostly of rock? Why?” • Integration questions across mass and distance slides – E. g., “Imagine a planet that is smaller than Earth and that was located 1.5 AUs from its sun, which is the same strength as the Earth’s sun. How would this planet’s potential for life compare to Earth’s?”

14. Figure 1. UCLA Study 1 - Interleaving by type of Posttest Integration Question.

15. Figure 2: UCLA Study 1 -Generation effects on single fact materials

16. Generation manipulations • Study 1 (UCLA): Sentence Completion – Generate: ____-type planets are mostly made up of gases. – Read: Jovian-type planets are mostly made up of gases. • Study 2, (UCLA; and UCB Classroom Studies 1 and 2): – Sentence level generation • E.g., “Describe in a sentence how the size of one planet's mass can affect another planet.” – Knowledge required for successful generation: • Mass or distance Only • Both mass and distance

17. Projects in Progress using WISE • Interactions between Generation & Interleaving (UCLA and Classroom) • Integration of M + D • Integration of M or D • Increase Contextual Interference (UCLA and Classroom) • Habitability and Detectability • Visual Support for Generation (Classroom) • Static versus Animated

18. Design Principle: Interleaving • Interleaving as spacing: Practical advantages • But are effects of interleaving more than the effects of spacing? • Contextual interference ideas (Battig, 1972, 1979) – Benefits dues to: • Reloading/ Reconstruction (e.g. Lee & Magill, 1983, 1985) • And/or Development of a higher order representation to differentiate interleaved materials (Shea & Zimny, 1983, 1988)

19. Disentangling Interleaving and Spacing • Initial experiment with second-language materials (courtesy of Hal Pashler): • Designed to: – Maximize contextual interference – Co-vary (i.e., unconfound) interleaving and spacing

20. Learning Materials English word “bird” “dege” “lind” Estonian Swahili -8 English words learned in both foreign languages (16 word pairs total) -6 repetitions (anticipation trials) of each word pair

21. Sample anticipation-method learning trial What is tree in Swahili? mufuma ?: The correct word is: mfufumaji

22. Sample Test Trial What is tree in Swahili? ?: in Estonian? ?:

23. INTERLEAVED A1 B1 A2 B2 A5 B5 A6 B6 A1 B1 A2 B2 A7 B7 A8 B8 A1 B1 A2 B2 A5 B5 A6 B6 BLOCKED A1 A2 A3 A4 A5 A6 A7 A8 A1 A2 A3 A4 A5 A6 A7 A8 A1 A2 A3 A4 A5 A6 A7 A8 BLUE RED PURPLE same spacing as in blocked double spacing vs. blocked double spacing vs. blocked

24. 90 80 70 60 % Correct 50 40 30 Interleaved Double Spacing 20 Interleaved Same Spacing 10 Blocked Same Spacing 0 1 2 3 4 5 6 test Learning Trials

25. 90 80 70 60 % Correct 50 40 30 Interleaved Double Spacing 20 Interleaved Same Spacing 10 Blocked Same Spacing 0 1 2 3 4 5 6 test Learning Trials

26. Implications and necessary next steps • Interleaving can cause intrusions and errors during learning that then foster long-term retention--and, possibly, transfer • The effects of interleaving and spacing – May be independent and additive, – But contextual interference (competition) maybe be necessary to demonstrate benefits of interleaving that go beyond the benefits of spacing • And, from an educational standpoint, it is essential to see whether the same pattern obtains with more complex and cumulative materials

27. Our thanks to  The Institute for Education Sciences and the Cognition and Student Learning program  Other members of the IDDEAS research team: – Lindsey E. Richland, Ph.D, (UCLA) – Britte H. Cheng (UC Berkeley) – Jason R. Finley (UCLA) – And a number of undergraduate students, especially Jeff Beyers, Fernando Cervantes, and Alexandra Hessenius • Relevant Links – IDDEAS : http://www.psych.ucla.edu/iddeas – WISE : http://wise.berkeley.edu – SCALE: (Synergy Communities: Aggregating Learning about Education) - http://scale.soe.berkeley.edu – TELS: (Technology Enhanced Learning in Science) - http://www.telscenter.org