Making Assessment Count (MAC) Consortium

1. University of Westminster • Gunter Saunders • Mark Clements • Peter Chatterton • University of Bedfordshire • Mark Gamble • City University • Kate Reader • University of Greenwich • Mark Kerrigan • Simon Walker • Cardiff Metropolitan University (UWIC) • Loretta Newman-Ford • University of Reading • Maria Papaefthimiou • Making Assessment Count (MAC) • Consortium • An informal group further developing and adapting best practice in the use of technology to enhance feedback. • Realising the benefits of the JISC funded project Making Assessment Count (Westminster) MAC Web-site https://sites.google.com/a/staff.westminster.ac.uk/mac • JISC Design Studio pages: • MAC http://jiscdesignstudio.pbworks.com/w/page/23495173/Making%20Assessment%20Count%20Project • MAC Consortium http://jiscdesignstudio.pbworks.com/w/page/33943261/Making-Assessment-Count-Consortium

2. The original MAC • Challenges being addressed: • Mismatch between feedback provided and students saying no feedback received • Students often do not make use of their feedback • Focused on the mark • Therefore do not derive benefit from the feedback in relation to future work MAC Framework for action on feedback (SOS model) E-Reflect software • Help the student strategically reflect on what they did and the feedback received • Connect the student’s feedback and reflections to the personal tutor • Encourages staff and students to engage with action on feedback • Link action on feedback to the personal tutorial system S = Subject O = Operational S = Strategic

3. MAC process – module assessment activities Student Subject tutor E-Reflect tool (Westminster online tool) Personal Tutor OR Subject Tutor Response to self-review questionnaire Learning journal entry Coursework & submission Dialogue Subject Marking & feedback Operational Automated feedback Strategic Feedback on the learning journal entry Dialogue

4. E-Reflect Tool • What is e-Reflect http://youtu.be/4_67G79CmHU • Student completing questionnaire http://youtu.be/PVeLndwdEFY • Student completing reflection http://youtu.be/WPBQ2AvnL04

5. MAC process – module assessment activities Alternative – e-Reflect used before coursework submission Student Subject tutor E-Reflect tool (Westminster online tool) Personal Tutor OR Subject Tutor Coursework & response to self-review questionnaire Learning journal entry Coursework submission Dialogue Subject Marking & feedback Operational Automated feedback Strategic Feedback on the learning journal entry Dialogue

6. MAC process – through a programme Module 1 Module 2 Module 3 Module 4 ……………………………………….. ……………..subject tutors……………. student personal tutor tutorials Learning Journal

7. Benefits • Staff • personal tutors provided with more timely information on their tutees overall performance • Clear framework for provision of feedback • Student • Additional layer of feedback facilitating strategic reflection • Strategy for using and acting on feedback • Institution • Raising of the profile of feedback (what it’s for and how it should be exploited within a broader assessment strategy) • Support for shifting of emphasis of personal tutorial to academic performance

8. Challenges • Making e-Reflect sufficiently flexible for a breadth of subject disciplines (including IT support) • Convincing other academic areas to ‘find the time’ for implementation • Integrating with/relating to similar ‘local’ approaches

9. MAC Consortium Progress • Cardiff Met – piloted with up to 350 Sports Medicine undergraduates and in School of Education • Greenwich – used across all Schools through training courses in learning and teaching • Bedfordshire – Just started using in Health Sciences and in IT/Business • Reading – Pilot with over 100 final year Life Sciences project students • Westminster – used now to support feedback on written exams in Life Sciences • City – pilot in Moodle with Politics and Sociology undergraduates

10. Why Evaluate E-Reflect and MAC at City? • Encouraging our students to reflect and learn from their feedback • May improve student retention • To engage more students with the personal tutorial system at City • Encouraging students to take a feedforward approach to work

11. How did we evaluate the E-Reflect tool and MAC Project? We took an ‘integrative’ evaluation approach (Cook, 2002) whereby we looked at how eReflect could be best integrated with other resources available to our users. In our case, the resource was our new virtual learning environment Moodle. • Students – how easy is eReflect to use? How enjoyable is it to use? Does eReflect feel personalised to me and meet my individual needs? Will eReflect help me to perform better in my studies? • Personal Tutors (teaching staff) – how easy is eReflect to use? How much additional work will I have to do over what I am doing already with my personal tutorials to use it effectively? Does eReflect look professional and does it integrate with the course content available on Moodle? • Support Staff – what are the hardware implications of eReflect? Who hosts it? What are the staff development and training implications of eReflect? • Developers – can eReflect be customised? Is it open-source and can we adapt it to better suit our needs? How do we report problems with bugs in the code? • Managers – how does eReflect fit within the institutional strategy? Can it enhance the status and attractiveness of the institution? What are the costs of running eReflect?

12. Evaluation Results E-Reflect Tool: • It would not be feasible to introduce an outside system to our students and staff for the following reasons: • Separate logins needed for staff and students from our institutional systems • Reliance on an external organisation for help and support with no formal service contract MAC Project: • The SOS model would be adopted, but delivered through our institutional VLE moodle • We would rename the project “Feedforward” in order to address issues with Feedback

13. Building the SOS Model into Moodle...

14. Feedforward at City • A very small scale pilot was run in the Politics and Sociology departments this year to test if moodle was a suitable way to implement the SOS model • Politics Department: we piloted Feedforward as part of an optional exercise during study skills week. • Sociology Department: we piloted Feedforward in one module taken by first year students on one assignment. • Lessons learnt – this needs to be embedded in a program with engagement from all academic staff in the department, as an optional small scale pilot there was little uptake by students. • The technology was not a barrier to use, those students that engaged with the activity navigated through the system effectively.

15. Future Plans... • The MA in Academic Practice will be implementing Feedforward in their Student Support module next year. • Arts and Social Sciences will be investigating the possibilities for using the SOS model to support students with English as a second language. • We are looking for other departments and programs at City University that would be interested in implementing the Feedforward model for feedback.

16. Further Information Making Assessment Count Practitioner Toolkit Making Assessment Count Evaluation MAC Video Case Study What students say about MAC E-Reflect videos • What is e-Reflect • Student completing questionnaire • Student completing reflection • Tutor commenting on student reflection • Tutor creating en e-Reflect questionnaire