Undergraduates as Teaching Assist a nts

1. Undergraduates as Teaching Assistants http://www.doc.ic.ac.uk/lab/firstyear/envision/ Reducing the marks-based culture of learning Tony Field Department of Computing Joint work with Esat Alpay, Peter Cutler, Susan Eisenbach

2. The “marks-based” culture • Use the weighting of the work to determine the time spent • Something that carries little or no weight gets little or no time • Concerns for marks rather than the knowledge/skills attained

3. The idea • Zero-weight first year tutorial work in Programming and Mathematics • 6/7 students • Weekly exercises (previously assessed by academic) • Pay A-grade 3rd/4th year undergraduates (UTAs) to help run the tutorials • UTAs mark exercises (feedback only) • Feedback, problem-related and general discussion Academic’s role: Mentor the UTA, check the marking, help plan meeting and guide discussions…

4. Some questions • Does zero weighting affect student motivation and participation? • How do students react to having tutorials run by peers? • What are the benefits for the UTA? • How do the academics perceive the change? • Is there a significant difference in student performance? • (Does it reduce the instances of plagiarism?)

5. Evaluation • Cost: Around £60-£75 per student per term (one tutorial/week, 2-3hrs marking) • Pilot study sponsored by Envision for Maths tutorials (PMTs) • 2-year study, with qualitative and quantitative evaluation: Changing the marks-based culture of learning through peer-assisted tutorials, European Journal of Engineering Education, Volume 35, Issue 1 March 2010 , pages 17 - 32

6. Questionnaires • One set for first years, the other for both UTAs and academics • Five-point Likert scale • Open-ended questions at the end

7. Main theme of student questions • How the first years compare the UTAs and the academics • ME – motivation and engagement • SD – skills development • ECF – explanation, contextualisation, and feedback

8. Evaluation of PMT and UTA. UTA PMT 1) Understanding my needs as a first-year student 1 2 3 4 5 1 2 3 4 5 2) Getting me to participate in the PMT 1 2 3 4 5 1 2 3 4 5 3) Making the PMT work interesting 1 2 3 4 5 1 2 3 4 5 4) Making it easy for me to talk about things I don’t understand 1 2 3 4 5 1 2 3 4 5 5) Having a difficult concept explained to me 1 2 3 4 5 1 2 3 4 5 6) Giving useful feedback on my work 1 2 3 4 5 1 2 3 4 5 7) Clarifying/correcting misunderstanding 1 2 3 4 5 1 2 3 4 5 8) Helping me improve my logic skills 1 2 3 4 5 1 2 3 4 5 9) Helping me improve my communication skills 1 2 3 4 5 1 2 3 4 5 10) Exploring a problem/concern thoroughly 1 2 3 4 5 1 2 3 4 5 11) Motivating me to want to do well on the course 1 2 3 4 5 1 2 3 4 5 12) Giving me reassurance about my learning progress 1 2 3 4 5 1 2 3 4 5 13) Clarifying why the material will be useful later on 1 2 3 4 5 1 2 3 4 5 14) Helping me to manage my time so that I can finish the exercises 1 2 3 4 5 1 2 3 4 5 15) Making me feel part of the department 1 2 3 4 5 1 2 3 4 5 16) Exploring related topics not covered directly by the course 1 2 3 4 5 1 2 3 4 5 ME Getting me to participate in the tutorial Making the tutorial work interesting Motivating me to want to do well in the course etc. SD helping me to…improve my logic skills ...improve my communication skills …manage my time so that I can finish the exercises etc. ECF having a difficult concept explained to me giving useful feedback on my work clarifying why the material will be useful later on etc.

9. Summary of results

10. Three key questions • The most discriminatory questions (UTA>Academic): Getting me to participate in the PMT (ME) Giving useful feedback on my work (SD) Helping me to improve my <maths> skills (ECF)

11. Main theme of UTA/tutor questions • How UTAs and Academics judge the PMT system • UPE – UTA performance and effectiveness UPE1 (teaching qualities) UPE2 (social/affective aspects as peer) • UB – UTA personal benefits • US – Support received by the UTA

12. UPE1 The UTA prepares well for the PMT meeting The UTA uses initiative in running the meeting The UTA has a clear idea about how to run the meeting etc. A: Evaluation of PMT. 1) The UTA gets good support for his/her role 1 2 3 4 5 2) The UTA’s communication skills are improving over time 1 2 3 4 5 3) The UTA prepares well for the PMT meeting 1 2 3 4 5 4) The UTA is a role model for the first year students 1 2 3 4 5 5) The UTA has many interests in common with the students 1 2 3 4 5 6) The UTA takes the lead role at the PMT meeting 1 2 3 4 5 7) The UTA uses initiative in running the PMT meeting 1 2 3 4 5 8) The UTA gets good support from the academic (PMT) 1 2 3 4 5 9) The UTA gives thorough feedback to the students 1 2 3 4 5 10) The UTA explains the PMT material clearly 1 2 3 4 5 11) The students find it easy to talk openly to a UTA 1 2 3 4 5 12) The students find it easy to talk freely with the academic (PMT) 1 2 3 4 5 13) The students want the UTA to think well of them 1 2 3 4 5 14) The students want the academic (PMT) to think well of them 1 2 3 4 5 15) The students find it easy to ask a UTA when they don’t understand 1 2 3 4 5 16) Most students are more concerned with marks than understanding 1 2 3 4 5 17) I suspect that some of my students copy their PMT submissions 1 2 3 4 5 18) The students take the feedback on their work seriously 1 2 3 4 5 19) The UTA has a clear idea about how to run the PMT meeting 1 2 3 4 5 20) The UTA and academic (PMT) discuss teaching strategy before the meeting 1 2 3 4 5 21) I enjoy seeing the students develop their skills 1 2 3 4 5 22) The UTA is a student so it easier for him/her to explain concepts to the PMT students 1 2 3 4 5 23) The students have improved in the PMT meetings over the year 1 2 3 4 5 24) The mentoring role of academic (PMT) is very important for a UTA 1 2 3 4 5 25) The UTA finds it hard to get the students to participate 1 2 3 4 5 26) The academic (PMT) finds it hard to get the students to participate 1 2 3 4 5 27) The UTA finds it helpful to revisit the first year material 1 2 3 4 5 28) The academic (PMT) gives the UTA useful advice about running the PMT 1 2 3 4 5 29) The UTA learns useful transferable skills by helping with the PMT 1 2 3 4 5 30) Having a UTA helps enhance the department’s intellectual culture 1 2 3 4 5 31) The payment is the most important motivator for a UTA 1 2 3 4 5 32) The experience is the most important motivator for a UTA 1 2 3 4 5 UPE2 The students find it easy to talk openly to the UTA The UTA is a student so it’s easier for them to explain… The UTA is a role model etc. UB The UTA learns useful transferable skills The experience is the most important motivating factor The UTA finds it useful to revisit the material Etc. US The UTA gets good support from the academic The mentoring role of the academic is important The academic gives the UTA useful advice etc.

13. Summary of results (17/23 returns) Mean Standard deviation UPE1 UTA 4.02 0.45 PMT 4.34 0.56 UPE2 UTA 3.85 0.69 PMT 3.67 0.80 UB UTA 4.08 0.64 PMT 3.90 0.57 US UTA 3.42 0.72 PMT 3.86 0.79

14. Four key questions • Statistically significant disagreements (UTA vs Academic score): • The UTA prepares well for the PMT meeting (4.06 vs 4.59) • The UTA takes the lead role at the PMT meeting (4.24 vs 4.88) • The UTA and academic discuss teaching strategy before the meeting (2.45 vs 3.47) • The UTA is a student so it is easier for him/her to explain concepts to the students (3.88 vs 2.94)

15. Quantitative Measures UTAs mark rigorously!

16. Open-ended questions Observation 1: Students really like it • The UTA scheme is the best part of the whole non-programming oriented part of the course! • Excellent scheme, that a lot of other courses do not have, has helped a lot in studies, and to settle in • overall extremely helpful...don't know what I would have done without it! • Best method of study by far • I think the PMT sessions is the most helpful hour of the week. • very helpful, can't think of any way to gain significant improvement. • The UTA scheme was the most helpful learning tool in my course this year. • Excellent idea. It both encourages us to aspire to work hard and helps the UTA in furthering his/her skills in teaching. • I really appreciated that the student who is still studying at the Universrity and has very recent experience with the functioning of the course is very helpful. • Execllent, everybody wins. • I think it's fantastic. Great job. Really appreciate the "anytime" approach. • I think it is better than the tutorials. • Its an excellent system that allows people to fully grasp concepts on the course. • Personally I find it very helpful and my UTA is very keen to explain anything we ask. Thumbs up for UTAprogramme. • It is very helpful as unlike lectures, the UTA understands what areas we are unsure on, since he is student after all! • In my specfic case, my UTA is outstanding. He is a superb student, and has a lot of time and effort for me. • My UTA was very helpful in this course. • Excellent. Great help. I've spoken to first year maths degree kids and they are jealous of the amount of help I get in smaller groups. I have found the PMT very very helpful indeed and would like even more of this time set aside for us. We have many hours of contact time each week for programming related help, yet only this one hour PMT slot for maths (which I and others think is the more difficult of the sections). • Good idea, provides some useful feedback • Excellent scheme, that a lot of other courses do not have, has helped a lot in studies, and to settle in • I think it's fantastic. Great job. Really appreciate the "anytime" approach. • The UTA scheme is the best part of the whole non-programming oriented part of the course! • Best method of study by far • etc.

17. Open-ended questions Observation 2: They like students as TAs • It's a "family" atmosphere • The environment feels a bit more comfortable • He is also a student, so he can give tips of what exactly is expected from us • Down to earth, jokes, knows the situation that we are in and can relate to it. • He's friendly and helpful and gives good explanations of the material • The opportunity to have somebody who is more on our wavelength • Relaxed learning • It is personal, small group is good for learning, get to know everyone in your group, don't feel scared by results, can discuss work with all members • That it is mainly run by student - in the way that only person who is still studying can do • They know exactly what they found difficult when they did the course and guide according to that. • Very friendly and straight to the point • It's very casual and he not only understands my problems, but also anticipates them. • That the UTA is run by students of approximately my age • UTA's are closer to our age and I think are better often at communicating with us than the lecturers • I believe its helpful, since sometimes you can approach the UTA in a more open manner than you would to with the tutor and that is sometimes helpful. • It's a "family" atmosphere • They know exactly what they found difficult when they did the course and guide according to that • The opportunity to have somebody who is more on our wavelength • UTA's are closer to our age and I think are better often at communicating with us than the lecturers • etc.

18. Open-ended questions Observation 3: Some UTAs need vetting/better training • More UTA involvement in the tutorial if possible • The UTA should be commanded to make students/tutees participate in the session • A more lively atmosphere would be nice • MAKE IT MORE FUN!!! • Better communication • More passionate uta's • Train them more in terms of dealing with students beforehand • It is a good system, my UTA just happens to be a little too shy for the job • Train the UTA • Possibly a more careful choice of UTA candidates • It is good if the UTA doesn't keep canceling the meetings/ not turning up to them. • My UTA for the PMT course was incredibly bad at 'public speaking', incredibly nervous etc. • It works fine, but some people get a poor UTA, which gives them a disadvantage • Train them more in terms of dealing with students beforehand • Possibly a more careful choice of UTA candidates • More UTA involvement in the tutorial if possible • A more lively atmosphere would be nice • etc.

19. Open-ended questions Observation 4: Staff need better training(!) • PMT should do more explaining, not just UTA • Acedemic teachers should involve more in the meeting. • Lecturers that are assigned to the PMT groups could be more involved with the PMT • My PMT did not turn up to most of my tutorial classes, which was disappointing • The academic staff should get more involved • Generally, the member of academic staff doesn't really do anything. • The main PMT tutor never participates on the scheme. Only the UTA is involved. • We end up with pmt sessions where UTAs do all the work while tutors are not participating. • PMT is not always there, and when is is usually working and not joining in • It was working fine until he stopped showing up a month ago • The academic staff should get more involved • My PMT did not turn up to most of my tutorial classes, which was disappointing • etc.

20. Open-ended questions Observation 5: UTAs benefit substantially • The meetings are fun and it is good talking with the students and getting them involved in the subjects. • Other first years ask me questions in the labs - I like helping people and it is good to know you've done it. • The experience and confidence it has given me, and watching how some of the students take the sessions on board and really improve. • Enjoyable experience, learned a lot myself, good contact to PMT. • Interacting with first year students, The experience of teaching. • I enjoy working with my PMT (Dr. X) very much. • Good teaching experience. • Experience gained - teaching, assessing. Seeing students develop/overcome learning barriers. • To try out teaching - A chance to improve my presentation skills - Be able to explain topics clearly - Watching students learn • Many people benefit (student, UTA, PMT). Good atmosphere. • The UTA program is beneficial to all of the three parties: • Students have the opportunity to participate in teaching roles which would be a valuable experience and it also offloaded some workload for PMTs. • The UTAs gain valuable experience (and some extra money which is always useful!). • Enjoyable experience, learned a lot myself, good contact to PMT. • Other first years ask me questions in the labs - I like helping people and it is good to know you've done it. • The experience and confidence it has given me, and watching how some of the students take the sessions on board and really improve. • The UTAs gain valuable experience (and some extra money which is always useful!). • etc.

21. Open-ended questions Observation 6: Academics love it! QUIZ: Why?

22. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • I no longer have to mark the homework

23. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • They do the marking! • I no longer have to mark the homework • They do the marking!

24. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • They do the marking! • No more marking. • I no longer have to mark the homework • They do the marking! • No more marking

25. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • They do the marking! • No more marking. • Not doing the marking. • I no longer have to mark the homework • They do the marking! • No more marking • Not doing the marking

26. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • They do the marking! • No more marking. • Not doing the marking. • Avoiding the marking! • I no longer have to mark the homework • They do the marking! • No more marking • Not doing the marking • Avoiding the marking!

27. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • They do the marking! • No more marking. • Not doing the marking. • Avoiding the marking! • No marking. • I no longer have to mark the homework • They do the marking! • No more marking • Not doing the marking • Avoiding the marking! • No marking

28. Open-ended questions Observation 6: Academics love it! • I no longer have to mark the homework. • They do the marking! • No more marking. • Not doing the marking. • Avoiding the marking! • No marking. • I appreciate not having to do the marking or prepare in detail for the tutorial. • Avoid the marking of PMT on weekly basis. • It saves a lot of time that I can more profitably spend on other activities - research for example. • More fun, less work, not so much boring exercises. • Not having the tedium of explaining this material for the 20th time. • It takes some of the teaching burden away from me, and made it easier to interact with the students on a personal level. • I no longer have to mark the homework • They do the marking! • No more marking • Not doing the marking • Avoiding the marking! • No marking • It takes some of the teaching burden away from me, and made it easier to interact with the students on a personal level • etc.

29. Conclusions • From the student perspective, the UTA performance exceeds that of the academic • Despite individual problems, students value the system and see UTAs as role models • The system fosters a sense of community; UTAs become mentors (and ‘buddies’?) • Academics universally praise the UTA system • There is no significant change in student exam performance, attendance etc. • Needs UTAs to be carefully chosen & trained and academics to engage fully • Successfully instils a culture of learning based on the intrinsic value of material, rather than credit