A View on the State of CS Education Research

1. A View on the State of CS Education Research • Mark Guzdial • Professor, School of Interactive Computing • College of Computing • Georgia Institute of Technology 1

2. Story • Brief history of a 40 year old field: From ESP to ICER • Themes: • The Sorry State of CS1 • Prior Conceptions in/of CS • How to Improve CS Ed, and the Research Needed • Future Directions sprinkled throughout

3. The Golden Age of CS Ed Research • Late 1960’s to 1990: • Seymour Papert, Andrea diSessa, Elliot Soloway, Ben Shneiderman, John Anderson,Marian Petre, T.R.G. Green • Soloway & Spohrer “Studying the Novice Programmer” • Empirical Studies of Programmers and PPIG Workshops • Then NSF and ONR decided empirical research wasn’t resulting in faster programmer development. • All funding ceased. Nearly all US CS Ed research disappeared.

4. ACM International Computing Education Research Workshop • Created as an outlet for the “Bootstrapping” and “Scaffolding” efforts to re-create CS Education Research in the United States. • Funded by National Science Foundation (Andy Bernat) • Josh Tenenberg (U Wash) with Sally Fincher and Marion Petre • Built around Multi-Institutional, Multi-National (MIMN) studies. • Why do things go wrong? Because of teacher? School? Approach? • Or maybe because learning CS is just hard? All ICER Papers can be found at http://www.acm.org/dl

5. Building from Teaching to Research All ICER Papers can be found at http://www.acm.org/dl

6. The Sorry State of CS1 • Soloway’s Rainfall Problem • 14% of CS1 students at Yale can solve it • McCracken study “Build a Calculator” (first MIMN study) • Average score: 21% • Lister (Leeds) study: Array and loops MCQ • Average score: 60% • 23% of students got less than 5 problems completed.

7. Allison Elliot Tew’s dissertation • Allison Elliott Tew has just completed the first language-independent validated test of CS1 knowledge. • 950 subjects, 3 schools, 2 countries. • Average score on pseudo-code: 33.78%On native code: 48.61% 49

8. What’s so hard? • Brian Dorn’s graphics designers (more tomorrow) ranked “assignments” as one of the hardest ideas to understand. • Allison’s toughest problem: A=<x>, B=f(A), what can we say about A and B? • Middlesex University found that a test of assignments and sequence was a nearly perfect prediction of CS1 performance. • ==> The simplest stuff is much harder than we realize.

9. What do students know about computing before they start? (Part 1)

10. What do students know about computing before they start? (Part 2)

11. What do students know about computing before they start? (Part 3)

12. Future work: Mechanisms and Efficacy • Research question: How do novices explain the computing in their lives? • How does Google work? How does a font get rendered? • Research question: How much does learning CS influence everyday use of computing?

13. Whether “Objects-first” really matters

14. There is no “first” in CS Education • Constructivism: People are not a blank slate. • Students already “know” a lot about computation entering CS1. • Have to build on where they are, not where we want them to be. • Changing prior conceptions is very hard. • Problem-based learning and contextualized computing education work, because people can learn partial and out-of-sequence information.

15. Current PhD Research: Mike Hewner • High school students cling to prior conceptions of CS despite interventions. • “Someone who can make magazine covers in Photoshop is a really great computer scientist.” • Science educators address this problem, and know what to look for. • What are students’ conceptualizations of CS? • Productive vs. Potentially Problematic Conceptualizations

16. Improving the state of CS Ed • People decide early that computer science is the pits, and they never change their mind. • Yardi and Bruckman found this attitude in pre-teens (ICER 2007) • Dorn (ICER 2010) found the same in adults. • P2: I went to a meeting for some kind of programmers, something or other. And they were OLD, and they were nerdy, and they were boring! And I'm like, this is not my personality. Like I can't work with people like that.

17. Attitudes aren’t coming from CS Class Can’t be. There isn’t one.

18. Growing CS Ed in Secondary Schools • The US education system is locally controlled. • Can’t fix things nationally.Have to fix things 51 times over. • You can’t create a demand for CS teachers, without CS classes to hire them into. • Students won’t take a CS class that doesn’t “count.” • Hard to retain and improve CS teachers without certification.

19. The new AP CS and CS10K • The US NSF is funding a new Advanced Placement exam “Computer Science: Principles” • One way to change 50 states at once. • The College Board is non-profit, but has to break even. • For the new AP CS:P to be solvent, must have 20,000 test takers. • NSF’s CS10K Goal: To have 10,000 AP CS:P high school teachers by 2015.

20. Current PhD Research: Lijun Ni • Teachers who develop teaching identities are retained and improve. • Much of teacher identity is based on certification. • In general, 46% of STEM teachers leave in first five years. • How many of those 10K teachers will be left in 2020? • How do CS teachers develop a sense of identity? • Lessons learned: Merged identity not new, and Community helps.

21. Future Research: Phonics of CS Ed • Students are confused about the details of CS Ed. • How assignments work. • How to read code. • How to sequence code. • The difference between IF and WHILE. • Objects, design, recursion, data structures: All problems, but maybe smaller if we got the simple stuff better. • Context motivates. But we need to teach better.

22. Powerful idea we don’t use: Worked Examples • Sweller et al. taught algebra two ways: • One group: Saw couple problems worked out, solved 7 problems. • Second group: Saw couple problems worked out, saw other 7 problems also worked out. • Take 1/3 as much time. • On algebra test involving problem solving, groups perform the same. • Pirolli used worked examples to teach recursion. • Showed 8 problems for each of 7 LISP primitives. • Others (Atkinson, Derry et al., Catrambone et al.) show that practice is critical to learning from examples.

23. Powerful idea we don’t use: SBF • Ashok Goel has been studying for 20 years the knowledge designers have • Developed Structure-Behavior-Function models. • Structure: The Code. • Function: What it achieves. • Students tend to see structure, and know function. • Behavior: What the code does to achieve function. • We don’t teach behavior.

24. Need a new medium for CS Ed • Textbooks can only show structure and identify function. • Visualizations can show behavior, but we know that it won’t be learned unless students demonstrate it. • We in US need a new medium for learning CS in a distance learning format, for CS10K.

25. Questions?