SIG3.5 – SGs for Humanities & Heritage user studies

1. SIG3.5 – SGs for Humanities & Heritageuser studies MichelaMortara, Chiara Eva Catalano, CNR – IMATI Jeffrey Earp, CNR-ITD

2. Overview Icura [ongoing, 50%] Playing History – The Plague [started, 10%] • Playing History – The Plague [started, 20%]

3. User study #1: Icura

4. Game description • Educational objective: cultural awareness - Japanese etiquette • Genre: adventure • Environment: 3D • Platform: PC • Intended educational context: informal (at home) • Developer: TU-Wien • Duration: less than 1 h • Target: secondary school and higher • Previous studies: evaluation done by the developers at the Wien university on a group of 20 students. Reports on papers and PhD. thesis

5. Previous evaluation • Pre-questionnaire (demographic data & game habits) • Pre-test with 12 questions ( on language basics, behaviour & etiquette, culture and society) • Game session • Post-test (same 12 questions) • Final questionnaire about user satisfaction Observations: • On average from +5 to +10 correct answers btw pre and post • Selective attention bias • Learning gain linked to the way the content is communicated (information agent vs quest) • Lacking estimate of retention

6. Our setting • Same phases but different questions in the pre-test to avoid the selective bias • Higher participant number (CNR, RWTH, ORT, DIBE) • Open answers to assess higher level in Bloom’s taxonomy (post questionnaire) e.g.: • What have you learnt from the game and how (through what mechanics)? • Did the game increase your interest towards Japan/ Japanese culture? Would you read more about Japan? Would you visit it? • How would you summarise Japanese culture in 3 adjectives? • What can you deduce about Japanese culture from the facts you learnt in the game? • How would you compare Japanese and Western culture? • Plan to repeat the post test after a few months to evaluate retention

7. Current status • CNR session • 10 university students • Colleagues&friends • RWTH session • 17 university students • ORT session • 12 people • students • DIBE session

8. Partial outcomes

9. Partial outcomes

10. Partial outcomes

11. Playing history – the plague

12. The game • Educational objective: history, historical reconstruction • Genre: adventure • Environment: 3D • Platform: browser (Unity Web player) • Intended educational context: formal (at school) • Developer: Serious Games Interactive • Duration: nearly 3 hours • Target: 8-13 • Previous studies: evaluation done by a master student with two primary school classes in Denmark. Report in her Master thesis (in Danish).

13. The experience: a Joint Research Action • JRA involving CNR-IMATI (SIG3.5), CNR-ITD (TC2.8, T6.3), SGI and DIS (Deledda International School) in Genova • Objectives: • explore the suitability and effectiveness of serious games in the field of humanities and cultural heritage in schools; • evaluate the educational efficacy of the game and the level of student engagement and enjoyment triggered by the learning activities; • investigate critical aspects of SG deployment in formal ed./school contexts, especially teacher’s role • examine cultural/educational issues related to game’s adoption in Italy (game produced in northern Europe and set in medieval Italy)

14. The context • School: English-speaking school in Genoa following International Baccalaureate & IB Middle Years programmes • Classes: 2 classes Year 7 (11-12 year olds). Approx. 35 pupils • Deployment settings: computer lab, classroom, home access

15. Research framework • Approach: qualitative, design based research – DBR Collective • Tools: • question matrix for gathering teachers’ GBL/ICT experience & attitudes, plus their expectations, intentions and goals for the experience • pedagogical planning tool (teachers document intentions/goals of intervention and plan learning activity sequence) • researchers’ observation sheets for monitoring classroom activities • pupils’ pre/post tests (in teacher’s resource pack from SGI) • mind map recording pupils’ collective pre/post knowledge/awareness • teachers’ monitored focus group • teacher questionnaire • reflection sheet for recording researchers’ attitudes, intentions, goals and interventions in the experience

16. Activity Phases 1 & 2… 1. Teacher preparation • introduction to the experiment, the game and its teaching resource pack • teachers’ group discussion of matrix questions (researcher compiled) • tech. training session (game’s teacher functions; ITD pedagogical planner) • teachers consolidate pedagogical plans • Deployment • pupil’s individual pre-test & collective brainstorming of pre-knowledge (making collective mind map) • class briefing: game presentation + play • gameplay sessions in pairs (plus possible home play) • pupil summative test & debriefing (re-examining collective mind map) • total 4/5 school periods with researcher monitoring & support

17. … Activity phases 3 - 6 • Teacher debriefing • focus group (inc. re-examination of matrix questions and pre-deployment responses, and of pedagogical plan/s) • final questionnaire • Analysis of data gathered from phases 1-3 • Follow up • Repeat of post test to evaluate pupil retention • Final data analysis and reporting

18. Current status • Series of organisational meetings held involving CNR, DIS & SGI • Activity phase 1 commenced • introduction to teachers • discussion / compiled of teachers’ of question matrix Issues with running game on DIS network have delayed commencement of deployment sessions with pupils • Solution: DIS agreed to move students to CNR (2 sessions in May)