Challenges in Computing Education Research: What We Need to Know to Reach Computational Thinking for All

1. Challenges in Computing Education Research: What We Need to Know to Reach Computational Thinking for All Mark Guzdial School of Interactive Computing

2. The Two Cultures

3. Story • Research Questions of Computing Education • Story 1: Teaching Computing to Computing-related Majors • Measuring knowledge, measuring naïve understanding, changing the task • Bottom line: It’s much harder than we think, and we don’t really know why. • Story 2: Teaching Computing to Everyone • Computing for All at Georgia Tech • 1999-2003: One course for all • 2003-2008: Contextualized Computing Education • Studying context and computing • Evaluation Results

4. Computing Education Research • Key Question: How do people understanding computational processes and devices; and how can we improve that understanding? • As a discipline, like other domain-specific education research fields. • Mathematics education, physics education, chemistry education, etc. • Applying the general findings of education research, with the insights of a domain, to explain learning about the domain.

5. The “Geek Gene” Question • What influences people’s understanding of computation and their ability to learn computation? • Why do so few people find programming easy to learn, but those that do find it very easy? • Hypothesis #1: Some people have the “geek gene” • Hypothesis #2: Some people accidentally get the right background. • Math education and “unitization”

6. Potential Causes for the “Geek Gene” • Experiences with time, process, representation, and agency • Spatial Intelligence (Shea & Lubinsky, 2001)

7. “Why Johnny Can’t Compute” • How should we be teaching computing? • What is a developmentally appropriate curriculum? • What are the factors that influence that developmental path, e.g., culture, motivation? • Multiple studies have sought to measure learning about programming in undergraduate years. • Results are always less than we anticipate • Yet, people seem to have significant naïve understanding of computing

8. Rainfall Problem: Elliot Soloway (1989) • Students were given some form of this problem: • Write a program that repeatedly reads in positive integers, until it reads the integer 99999. After seeing 99999, it should print out the average.

9. Results • Results at Yale: • CS1 Students: Only 14% correctly solved the problem (correct other than syntactic errors). • CS2: Only 36% got it correct. • Junior/Senior Systems Programming Course: 69% • This problem has been replicated at schools like University of Michigan and Georgia Tech • Always with the same results • Why is this hard?

10. Large MIMN Studies • McCracken ITiCSE Working Group 2001 • “Build a calculator.” • Five institutions, four countries • Average score: 22.89 out of 110 • Lister ITiCSE Working Group 2004 • Maybe the problem is design. • Using Multiple Choice Questions in a multi-institutional, multi-national study.

11. Lister Study

12. Naïve Understanding of Computing • Studies by Miller (1981) and Pane (2001) asked people to tell a person/computer how to solve some task. • Results tell us how people think about specifying process before learning a language. “Make a list of those employees who make more than 8 dollars/hr. and also are over 50years old. List should be organized by employee name.”

13. Naïve Computing • No one uses loops • Some IF’s, never ELSE’s • Preference for qualification instead of iteration • “For all the last names that start with ‘G’”… • No object-oriented programming • There is a sense of instances with properties • No notion of classes or methods. • All programs from perspective of programmer, never of agent/object • Heavy use of production rules, event-driven programming.

14. OPTION 1: if (value < 10) then value = value + 10; else sum = sum + value; end if. OPTION 2: if (value < 10): value = value + 10; not (value < 10): sum = sum + value; end (value < 10). Replacing ELSE’sThomas R.G. Green (1977)

15. Parallelism in Commonsense Computing Suppose we sell concert tickets over the telephone in the following way – when a customer calls in and asks for a number (n) of seats, the seller 1) finds the n best seats that are available, 2) marks those n seats as unavailable, and 3) deals with payment options for the customer (e.g. getting credit or debit card number, or sending the tickets to the Will Call window for pickup). Suppose we have more than one seller working at the same time. What problems might we see, and how might we avoid those problems? (Lewandowski et al. 2007)

16. Results:

17. Summary: CompEd Research Questions for Computing Majors • What is hard about programming? • How do we measure student understanding about computing? • Where is the challenge about our tools versus about computing? • Can changes to our tools make programming more obvious? • Can we build on students’ naïve understanding?

18. 1961 MIT Sloan School Symposium

19. Learn Programming to Re-Think Process Everywhere • Alan Perlis argued that computer science should be part of a liberal education. • Explicitly, he argued that all students should learn to program. • Why? • Because Computer Science is the study of process. • Automated execution of process changes everything • Including how we think about things we already know

20. The Power and Fear of Algorithms • The Economist (Sept., 2007) spoke to the algorithms that control us, yet we don’t understand. • Synthetic Biology, Credit Ratings, Adjustable Rate Mortgages, Search Rankings • C.P. Snow foresaw this in 1961. • Those who don’t understand algorithms, can’t understand how the decisions are made. “A handful of people, having no relation to the will of society, having no communication with the rest of society, will be taking decisions in secret which are going to affect our lives in the deepest sense.”

21. The challenge of teaching computing is not just about computing majors (Endy et al., Nature 2005)

22. Alan Kay’s Dynabook (~1970) • Alan Kay (2004 ACM Turing Awardee) sees the Computer as humanity’s first metamedium • A medium that can represent all other media. • Programming as an important new medium • The computer-as-Dynabook is for creative metamedia exploration and reading

23. Why should everyone learn computing? • Perlis: To have a powerful new tool to think with. • Snow: Because it’s necessary to participate and understand in the modern world. • Kay: To have full expressive power with the most powerful and creative medium humans have ever invented.

24. Context and Computation • Context: What students want to do with computation. An application domain. • Biology: Invent new medicines. • Engineering: See and test models before realizing them. • Media: Manipulate sounds and images to express new thoughts in new ways.

25. 2. Tools that can be seen from context as a lens 1. A subject that intersects a context 3. A lens that offers a new way of seeing and doing in other contexts. Relationship between context and computing Segregationist Integrationist Lewis & Smith, inroads, June 2005 Computing as literacy Synergist

26. Examples of Context and Computing • Tool Model • Bio2010 (US NRC) calls for programming and computational models for all Biology students. • Physics teaching VPython for labs where they solve three-body problems. • Lens/Literacy Model: • Jeanette Wing’s “Computational Thinking” • Computer science provides the tools and metaphors for understanding our world

27. Boxer Logo Idit Harel and Instructional Software Design Project (ISDP) Teaching Computing as Literacy

28. Emile Example: Teaching Computing as Lens on Physics Bruce Sherin

29. Implementing the Subject Model • Fall 1999: All students at Georgia Tech must take a course in computer science. • Considered part of General Education, like mathematics, social science, humanities… • Why did Georgia Tech make that decision? • Computing is important • Computing was a College. • Solved a problem for Engineering • Made a competitive distinction for Liberal Arts

30. One-class CS1: Pass (A, B, or C) vs. WDF (Withdrawal, D or F)

31. Contextualized Computing Education • What’s going on? • Research results: Computing is “tedious, boring, irrelevant” • Since Spring 2003, Georgia Tech teaches three introductory CS courses. • Based on Margolis and Fisher’s “alternative paths” • Each course introduces computing using a context (examples, homework assignments, lecture discussion) relevant to majors. • Make computing relevant by teaching it in terms of what computers are good for (from the students’ perspective)

32. Contexts, Results, and Themes • Media Computation • Computing for Engineers (MATLAB) • Robotics for CS1 • Institute for Personal Robotics in Education • Pre CS1 • Contexts with Girl Scouts • Beyond CS1 • Impact on CS2 • Gameboy programming for Computer Organization

33. Media Computation:Teaching in a Relevant Context • Presenting CS topics with media projects and examples • Iteration as creating negative and grayscale images • Indexing in a range as removing redeye • Algorithms for blending both images and sounds • Linked lists as song fragments woven to make music • Information encodings as sound visualizations 33

34. Results:CS1“Media Computation”

35. Voices from Media Computation Students • Intl Affairs student (female): “I just wish I had more time to play around with that and make neat effects. But JES [IDE for class] will be on my computer forever, so… that’s the nice thing about this class is that you could go as deep into the homework as you wanted. So, I’d turn it in and then me and my roommate would do more after to see what we could do with it.” • “I dreaded CS, but ALL of the topics thus far have been applicable to my future career (& personal) plans—there isn't anything I don't like about this class!!!" • "Media Computation is a CS class but with less severity. The media part of the class is extremely visually interesting. I would only take another CS class if it were Media Computation."

36. Survey One Year Later • 19% of respondents had programmed since class ended "Did the class change how you interact with computers?” • 80% say “Yes” • “Definitely makes me think of what is going on behind the scenes of such programs like Photoshop and Illustrator.” • 'I understand technological concepts more easily now; I am more willing and able to experience new things with computers now’ • 'I have learned more about the big picture behind computer science and programming. This has helped me to figure out how to use programs that I've never used before.’

37. Results at Gainesville College (Tew, Fowler, Guzdial, SIGCSE 2005) (Tew, Fowler, Guzdial, SIGCSE 2005)

38. Results at U. Illinois-Chicago (Sloan and Troy, SIGCSE 2008)

39. A Context for CS1 forCS majors: Robotics • Microsoft Research has funded the Institute for Personal Robotics in Education • Tucker Balch, Deepak Kumar, Doug Blank • Joint between Bryn Mawr and Georgia Tech • http://www.roboteducation.org • Goal is to develop a CS1 with robotics as the context. • Includes a camera and media computation functions

40. Some Attitudes Results • All students enjoyed the robot, were comfortable with it, and found it easy to get working. • Personalizing the robot improved the course, in students’ opinion. • Reported that the class was about computer science • Found homework challenging • BMC students did more on homework “because it was cool.”

41. W’s vs. F’s, statistically significant

42. Pre-CS1: “Georgia Computes!”Use in Girl Scouts Workshops

43. What about beyond CS1? • If students have a contextualized CS1, how do they perform in a context-free CS2? • Are students as well-prepared in CS2 after a contextualized CS1? • Is context useful in CS2 and beyond?

44. The Impact of Contextualized CS1 on CS2 enrollment Bryn Mawr College Data for 12 years of CS2 CS2 Data Structures Enrollment

45. Including last two years (since robots) CS2 Data Structures Enrollment

46. A Contextualized CS2: MediaComp Data Structures How did the Wildebeests charge over the ridge in Disney's "The Lion King"?

47. Research Question: Is context still useful in a second course? • Mixed model research design • Interviews informing whole-class survey • 11% agreed with “Working with media is a waste of time that could be used to learn the material in greater depth.” • “I didn’t take this class to learn how to make pretty pictures.” • A majority of the class (70%) agreed or strongly agreed that working with media makes the class more interesting. • 67% of the students agreed or strongly agreed that they were really excited by at least one class project • 66% reported doing extra work on projects to make the outcome look “cool.” (Yarosh and Guzdial, JERIC, Jan 2008)

48. Context: Programming a Gameboy • Traditional Computer Organization using an imaginary processor, vs.Computer Organization using a Gameboy • Not all the same topics (e.g., Gameboy graphics modes) • Common focus on C programming, bit manipulations, model of the computer (including memory and I/O) • Post-class survey comparison: No significant performance difference. Big motivation difference. • 100% of students in Gameboy class did extra on at least one homework, because it was fun. • 30% of traditional students never put any extra effort in (p=0.046). (Tew, Dorn, Leahy, Guzdial, JERIC, 2008)

49. Summary: What do we know about contexts for Computing Ed? • Including context has led to evaluation reports of: • Increased retention (in CS1, across multiple schools, with both majors and non-CS majors, across genders and ethnicities), • Some evidence of increased enrollment in CS2, and • Student reports of increased time-on-task, even beyond CS1. • Not all contexts work for all students. • Context is about personal engagement • Some students do just fine with computing Fibonacci numbers • Therefore, we need to offer multiple contexts, for all Snow’s “cultures”

50. Summary: Does Contextualized Computing Lead to Literacy Models? • The Computing-as-subject model is not viable for all students from a motivation perspective. • Programming is detail-oriented, unforgiving.It’s hard. • The Computing-as-tool model is motivating. • Maybe it enhances learning of the context, too. • Maybe it can be a step towards a literacy model. • Computing-as-literacy model requires an investment in becoming literate. • What motivates that investment?