Effective Instructional Conversations + TuTalk Instruction

1. Effective Instructional Conversations + TuTalk Instruction Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute

2. Introducing the Instructional Team! Carolyn Rose cprose@cs.cmu.edu Iris Howley iris@cmu.edu Rohit Kumar rohitk@andrew.cmu.edu Elijah Mayfield elijah@cmu.edu

3. http://www.cs.cmu.edu/~cprose/Summer09.html 2009 videos are on line! You might want to review or see what is happening in other tracks.

4. Design PrincipleKnow what problem you are trying to solve!!

5. Design Process • What problem are you trying to solve? • Formulate analysis scheme • When should you intervene? • Use technologies like TagHelper and SIDE to track interaction and trigger support • What should the intervention be? • Technologies like TuTalk can be used to offer support

6. What Makes Instructional Conversations Effective

7. Problems with the Process reduce learning Design Challenge: Collaborative Process Problems Hurt Learning Interaction Processes Cognitive Processes Learning

8. Support structure is designed to elicit certain interaction processes, so its success should be reflected in a difference in interaction processes Collaboration support increases learning by changing interaction Support Structure Interaction Processes Cognitive Processes Learning

9. But if the process difficulties are not the same between student populations, then the support needs to be different too. X Collaboration support increases learning by changinginteraction Support Structure Interaction Processes Cognitive Processes Learning

10. Goals of Scripted Collaboration “The basic goal of scripted cooperation is to increase achievement by having students perform cognitive activities that they might not otherwise do. In addition, scripted cooperation is expected to reduce the incidence of negative group processes because of the size of the cooperating unit and because of the explicit requirement to alternate roles.”

11. Script Based Collaborative Learning Support • Static support approaches provide structure to offer support for collaboration before it begins • Script based support increases effective argumentation (Weinberger, 2003)

12. Social Script

13. What is a Knowledge Construction Dialogue?

14. Atlas: Knowledge Construction Dialogues • Purpose: test “Ask, Don’t Tell” strategy in an ITS • How can we get the most bang for our buck? • How far can we go with finite state dialogue management? (Freedman,2000) • How far can we go with shallow concept grammars? (i.e., Glass, 1999) • Knowledge Construction Dialogues (KCDs) • Interactive directed lines of reasoning: analogies, concrete illustrations • KCD Authoring Tool Suite(Jordan, Rosé, and VanLehn, 2001) • 55 KCDs fully implemented and pilot tested in 3 months

15. Here are a few things to keep in mind when calculating acceleration for a body at rest. Acceleration is change in velocity over time. If velocity is not changing, then there is zero acceleration. Now, if for a finite time interval the velocity remains zero, then it is true that it is not changing. Therefore, when a body is at rest during a time interval, its acceleration within that time interval is zero. Tutor: Here are a few things to keep in mind when calculating acceleration for a body at rest. Acceleration is change of what over time? Student: velocity Tutor: Right. If the velocity is not changing, what is the magnitude of the acceleration? Student: zero. Tutor: Good. If velocity remains zero within a time interval, how much is it changing? Student: zero Tutor: Super. So if a body is at rest during a time interval, what is the... KCDs versus Minilessons versus Hints

16. Here are a few things to keep in mind when calculating acceleration for a body at rest. Acceleration is change in velocity over time. If velocity is not changing, then there is zero acceleration. Now, if for a finite time interval the velocity remains zero, then it is true that it is not changing. Therefore, when a body is at rest during a time interval, its acceleration within that time interval is zero. Tutor: Here are a few things to keep in mind when calculating acceleration for a body at rest. Acceleration is change of what over time? Student: velocity Tutor: Right. If the velocity is not changing, what is the magnitude of the acceleration? Student: zero. Tutor: Good. If velocity remains zero within a time interval, how much is it changing? Student: zero Tutor: Super. So if a body is at rest during a time interval, what is the... KCDs versus Minilessons versus Hints What if this answer was wrong?

17. Here are a few things to keep in mind when calculating acceleration for a body at rest. Acceleration is change in velocity over time. If velocity is not changing, then there is zero acceleration. Now, if for a finite time interval the velocity remains zero, then it is true that it is not changing. Therefore, when a body is at rest during a time interval, its acceleration within that time interval is zero. Tutor: Right. If the velocity is not changing, what is the magnitude of the acceleration? Student: I don’t know Tutor: Do you remember what acceleration means? Student: It’s change in velocity over time Tutor: Right. So then if velocity is not changing, what is acceleration? Student: Oh, zero. Tutor: OK. If velocity remains zero within a time interval, how much is it changing? … KCDs versus Minilessons versus Hints

18. Design Process Tips

19. Exploratory Learning Overview • (Charnay & Reder, 1986) compare Worked Examples, Tutorials, Problem Solving, and Pure Exploration • Worked examples mixed with problem solving was the best • (Touvinem & Sweller, 1999) Worked examples with problem solving beats Pure exploratory learning • (Klahr & Nigam, to appear) tutorial learning plus problem solving is more efficient than pure exploratory learning Only High Level Goals Given + help requests Worked Examples Tutorial Learning Problem Solving Goals Provided Negotiable Problem Solving Goals Only High Level Goals Given + hints Pure Exploratory Learning Pure Exploratory Learning

20. Exploratory Learning Overview • Smithtown work (Shute & Glaser, 1990, etc.) and Computer-Based Simulation Games (Leutner, 1993) involve only imposed high level goals • Learning about a model or Survival • (Leutner, 1993) demonstrates the importance of students actively requesting help • Smithtown adapted to individual student needs Only High Level Goals Given + help requests Worked Examples Tutorial Learning Problem Solving Goals Provided Negotiable Problem Solving Goals Only High Level Goals Given + hints Pure Exploratory Learning Pure Exploratory Learning

21. Exploratory Learning Overview • The difference between problem solving and exploratory learning is the level at which goals are dictated • Many state-of-the-art tutoring systems fall into the problem solving category where problem solving goals are dictated • In an exploratory design environment, high level goals are dictated (improve efficiency), but problem solving goals (i.e., add a reheat cycle) are negotiated • Tutorial dialogue allows the tutor to influence problem solving goals (addressing cognitive load issues) while offering instruction and while allowing students to actively explore Only High Level Goals Given + help requests Worked Examples Tutorial Learning Problem Solving Goals Provided Negotiable Problem Solving Goals Only High Level Goals Given + hints Pure Exploratory Learning

22. Analysis Specific Topics General Conceptual Topics Topic Analysis Reveals Important Patterns of Interaction • Significant correlation between topic coverage and learning • (R2=.715, N=21, p < .05)

23. Tutor: Let’s stop for a minute and reflect on what we have seen. Having seen how changing T-Max (i.e., the Maximum Temperature at the Heater Outlet) affects the efficiency of the cycle, do you think it is a good idea in general for the Maximum Temperature to be increased and kept high? Student: yes Tutor: Okay. Let’s think now about one important consequence of keeping T-Max high. What happens to the steam quality at the turbine exit if you increase the maximum temperature of the cycle? Student: Increases Tutor: Right. As increasing the temperature makes the efficiency and steam quality better, we should increase the maximum temperature of the cycle. But can we keep on increasing the maximum temperature at the heater outlet indefinitely, or are there limitations to keep in mind? Student: there are limitations such as material constraints Tutor: Great. we should try to keep T-Max (i.e., the maximum temperature at the Heater outlet of the cycle) as high as possible without risking safety concerns or exceeding the maximum temperature the material can withstand. Keeping T-Max high increases efficiency and keeps the steam quality high, which are both important. This is an important principle to keep in mind while optimizing the efficiency of your cycle. Example Implemented Reflection Dialogue

24. TuTalk Authoring Process

25. Scenario Goal1 (Start Goal) Goal1_Step1 Goal1_Step4 Goal1_Step2 Goal1_Step3 Pair • Scenario/ Script • Goal • Template • Step/Pair • Concept Initiation Response1 Response2 Response3 Say1 Say2 Sub Goal 1.1

26. Getting Started • Start by creating a new script • That script will contain all of the goals, templates, and concepts that you define • Could contain multiple ways of achieving the same goal (via alternative templates)

27. Directed Lines of Reasoning • Building Blocks: Pairs are composed of an initiation and a response Tutor: Here are a few things to keep in mind when calculating acceleration for a body at rest. Acceleration is change of what over time? Student: velocity Tutor: Right. If the velocity is not changing, what is the magnitude of the acceleration? Student: zero. Tutor: Good. If velocity remains zero within a time interval, how much is it changing? Student: zero Tutor: Super. So if a body is at rest during a time interval, what is the...

28. Building a Pair

29. Adding Phrases to Concepts

30. Elaborating a Template

31. Example Pair

32. Example Pair

33. Concept Manager

34. Scenario Goal1 (Start Goal) Goal1_Step1 Goal1_Step4 Goal1_Step2 Goal1_Step3 Pair Initiation Response1 Response2 Response3 Say1 Say2 Sub Goal 1.1

35. Previewer

36. Test Interface

37. Some tips • Better to have several short turns than fewer long turns • Try to build the learner’s/user’s vocabulary (shaping) • It may help you avoid unanticipated responses • Try to think about what is it that we are trying to elicit from the student when writing tutor turns (initiations) • Author as many possible responses as you can in the time available • Catch all makes the machine look dumb!

38. Questions?