Geeta Hitch (Senior Lecturer, Dept of Pharmacy) Janet Webber

The impact of case studies formatively and summatively assessed on students’ examination performance. Geeta Hitch (Senior Lecturer, Dept of Pharmacy) Janet Webber (Senior Lecturer, Dept of Physiotherapy/Principal Lecturer LTI)

Background • PMM module /2nd year MPharm degree • Previously taught as lectures only • Concern- ‘decontextualised’ as well as there was a divide between experience of learning and that as a practitioner

Aim • To investigate the theimpact of case studies formatively and summatively assessed on students’ examination performance

Seven Principles of Good Practice in ‘Higher’ Education (Chickering & Gamson, 1987). • Encourages contact between students and lecturers • Develops reciprocity and cooperation among students • Encourages active learning • Gives prompt feedback • Emphasises time on task • Communicates high expectations • Respects diverse talents and ways of learning

Method • Use of ‘hybrid’ PBL in PMM module • Bridge the divide between theory and practice; real-life situation problems. • Incorporation of constructive alignment (Biggs, 2000) in the use of problem solving exercise • exam question format is case study style • Inclusion summative assessment of presentation & poster • PMM module 50% exam (25% on case study style question); 50% CW (of which 5% was allocated to this assignment)

Method: encompassed Seven Principles of Good Practice in ‘Higher’ Education (Chickering & Gamson, 1987). • Students placed in groups of 6; communicated with peers online via Studynet discussion site • ‘Hybrid’ PBL- case study with structured questions • Tutor acted as facilitator • Students given a timeline • Constant feedback was provided online • Had an oral presentation /poster (10 mins) • Followed by Q&A • Assessed by panel of 3 lecturers • CW mark of 5% in summatively assessed cohort • Encourages contact between students and lecturers • Develops reciprocity and cooperation among students • Encourages active learning • Respects diverse talents and ways of learning • Emphasises time on task • Gives prompt feedback • Encourages contact between students and lecturers • Develops reciprocity and cooperation among students • Encourages active learning • Communicates high expectations • Respects diverse talents and ways of learning Exam question: 25% of the marks- had a choice of doing one of the 2 case study style based questions

Method • Cohorts in which study was carried out:

Data analysis • Overall examination performance between all 3 cohorts was compared- Statistical Package for Social Science (SPSS for MS windows version 16). • The Students t test was used to compare the probability level set at 5% i.e. P < 0.05. If the calculated p- value was below the threshold chosen for statistical significance of 0.05, then the null hypothesis was rejected in favour of the alternative hypothesis. • Therefore any P values less than 0.05 were regarded as significant. • Null hypothesis: assessed case studies should make NO impact on students’ final exam performance • Alternative hypothesis: assessed case studies should make an impact on students’ final exam performance

Data analysis • Questionnaires • A qualitative analysis of the student feedback questionnaire was also carried out from the academic cohorts of 2008-09 (‘formative cohort’) and 2009-10 (‘summative cohort’) • Group dynamics and the impact of inclusion of summative and formative assessment was examined as to whether it was a driving force in participation of group work.

Results: Percentage of student cohorts in 2007-08 (Control cohort), 2008-09 (formative cohort) and 2009-10 (summative cohort) showing marks range scored (%) in the final PMM Examination question.

Results Percentage of student cohorts in 2007-08, 2008-09 and 2009-10 scoring more than 40% in final PMM Examination question More than 65% of students have scored greater than 40% in their final PMM exam question in 2009-10 cohort (summative vs lectures only) p≤0.0001;

Results • No significant difference in exam performance for students when formatively assessed case studies were used (2008-09 cohort) and when lectures only were used (2007-08 cohort) to deliver the curriculum (student T test; p≤0.97). • Significantly better overall performance in the relevant exam question when the case studies were summatively assessed in comparison to formative assessment (student T test; p≤0.0005);

2007-08 (control cohort) Comparison of final exam performance in the same cohort between 2 different pharmacy modules (PMM- pharmaceutical Microbiology & Manufacture and MPP3- Medicines & Professional Practice Level 3)

2008-09 (formative cohort) Comparison of final exam performance in the same cohort between 2 different pharmacy modules (PMM- pharmaceutical Microbiology & Manufacture and MPP2- Medicines & Professional Practice Level 2)

2009-10 (summative cohort) Comparison of final exam performance in the same cohort between 2 different pharmacy modules (PMM- pharmaceutical Microbiology & Manufacture and MPP1- Medicines & Professional Practice Level 1)

Cohorts general performance in other examinations • All 3 cohorts appear to be performing at the same level in 2 sets of exams results compared. • This bears a significance in terms of inclusion of case studies because: • The overall performance appears to be similar in 2007-08 and 2008-09 cohorts. • The overall exam performance in case study style based question is performed significantly better in the 2009-10 summative assessed cohort.

Questionnaires • Response rate: • 65% from 2008-09 (n=123) (Formative cohort) • 73% from 2009-10 (n=132) (Summative cohort) • Enjoyed taking part in group work: • 75% of formatively assessed cohort - yes • 90% of summative assessed cohort- yes • 58% of students participated in equal input into group work when case studies were formatively assessed in comparison to 83% when summatively assessed. • When students were asked if they participated more when summative assessment took place, • 76% replied yes and 66% of these stated that they were driven by a genuine desire to study by participating in the case study. • The other 34% were just performing a task as it was summative and exam marks mattered more to them.

Questionnaires • 56% of the formatively assessed cohort felt that they learnt better in group work than working alone • 64% of the summative assessed cohort felt that they learnt more in group work than by working alone.

Questionnaires • Formative assessments were not viewed as important to the students in achieving their learning outcomes for a variety of reasons. In the case of this cohort (2008-09), several reasons were cited by students in the feedback questionnaire: • “Too many summative assignments to hand in and therefore formative assignments are my last priority” • “Many students tend to rely on others to do the group work and then when they are told to get on with it, the work they hand in is not reliable” • “Some students were clearly not willing to prepare or participate in the presentations as there was no marks awarded for this work and so the rest of the group ended up doing their part of the work”

Discussion • Gibbs, (1999) - “Assessment is perceived by students as the curriculum and as such the power of assessment needs to be used strategically to help students learn” • Biggs (2002) identifies this- states that “students will only learn what they think will be assessed on as opposed to what is required of them to learn in the curriculum”.

Discussion • General observations: inclusion of case studies does improve overall performance- students engage more effectively when learning is contextualised and is of relevance to practice • Students also perform better when marks are allocated to assignments adding to overall module mark • If marks are not allocated towards assignments and towards module marks in general, students can be reluctant to participate in CW elements of the curriculum.

Finally we conclude: “That by inclusion of the 7 principles of good practice in undergraduate education, we have shown in this small study that how we practice the teaching is as much as important as to what the students learn.”

Seven Principles of Good Practice in ‘Higher’ Education (Chickering & Gamson, 1987). • Encourages contact between students and lecturers • Develops reciprocity and cooperation among students • Encourages active learning • Gives prompt feedback • Emphasises time on task • Communicates high expectations • Respects diverse talents and ways of learning

References 1. Chickering, A. W. & Gamson, Z. F. (1987). ‘Seven principles for good practice in undergraduate education.’ American Association for Higher Education Bulletin, 39(7), 3–7. 2. Gibbs, G (1999) Using assessment strategically to change the way students learn, in: S. Brown & A. Glasner (Eds) Assessment Matters in Higher Education (Buckingham, SRHE & Open University Press). 3. Biggs, J. (2002) Aligning the curriculum to promote good learning, paper presented at the Constructive Alignment in Action: Imaginative Curriculum Symposium, LTSN Generic Centre, November 2002. Available online at: www.ltsn.ac.uk

Thank you Any questions?

Geeta Hitch (Senior Lecturer, Dept of Pharmacy) Janet Webber