Presentation Transcript

1. Work package 6:Integration into Educational ProcessesWork package leader UWS

   Professor Thomas Connolly Dr. Elizabeth Boyle Dr. Thomas Hainey
2. Reminder of WP6 partners and original allocation of tasks T6.1 User and education stakeholder requirements (14) (Task leader: UWS, Thomas Connolly (7)) INESC-ID, Joao Pereira (7) T6.2 Metrics for SG in education (16) (Task leader: TUG, Michael Kickmeier (5)) UNI-GRAZ, Elisabeth Friedrich (7) UNIGE, Francesco Bellotti (4) T6.3 Integration methodologies (15) (Task leader: CNR-ITD, Rosa Bottino, Michela Ott (6)) UCM, Pablo Moreno-Ger (8) UNIGE, Francesco Bellotti (1) T6.4 Community of teachers and tutors (10) (Task leader: TU DELFT, Igor Mayer (3)) UU, RemcoVeltkamp (3) HWU, Theo Lim (4) Total 55
3. Recommendations in the review report and major changes to work plan The reviewers recommend that we change our approach in WP6.1 from surveys to a meta-analysis as “survey research is a weak methodology”. The review panel “commends the notion of constructing a database to synthesise previous research and sees this as an important contribution to the serious games community but its value will be significantly diminished if a broad and deep look at established researchers around the world is not included.” We should take “a broad and deep look at established researchers around the world” including the work of Eva Baker, Sigmund Tobias and Dexter Fletcher.
4. Publications Tobias, S and Fletcher, J. D. (Eds) (2011) Computer Games and Instruction Girlie C. Delacruz Gregory K.W.K Chung and Eva L. Baker (2010) Validity evidence for games as assessment environments. National Center for Research on Evaluation, Standards, and Student Testing Baker, E. L., & Delacruz, G. C. (2008). A conceptual framework for assessment of learning in games. In H. F. O’Neil & R. S. Perez (Eds.), Computer games and team and individual learning (pp. 21–38). Amsterdam, Netherlands: Elsevier.
5. Other recommendations in the review report “Take advantage of the EU Lifelong Learning Grant award and build in plans to measure learning impact and outcomes as part of the development of that game and report the finds as part of this WP when they are available – the panel sees this as a high priority contribution that this WP could make to the network.” “Since WP3 reported minimal penetration of SGs in engineering and manufacturing, it might be worth exploring how SGs might be integrated into engineering education, especially at the secondary level where there is probably a reasonable chance for effective use. The panel recommends an effort be made to coordinate further exploration of how best to integrate SGs into education with WP3 – use games identified in WP3?” “Coordinate taxonomies with WP1 – do not use something that is not being used consistently throughout the project and by other WPs.”
6. Proposed year 2 activities for WP6.1 and WP2:Literature search and meta-analysis “A serious and formal meta-analysis associated with learning outcomes associated with serious games would be highly regarded by the global SG community” and “the review panel recommends taking up such an effort early in year two planning efforts”. The proposed meta-analysis of learning outcomes seems to fit better under task WP6.2 metrics than WP6.1. It is proposed that the meta-analysis would be a joint activity of WP6.1 and WP6.2 with UWS, TUG, UNI-GRAZ and UNIGE working together on the literature search and meta-analysis. INESC-ID to be redeployed in WP6.3?
7. Meta-analysis step 1: Formulation of the problem “A serious and formal meta-analysis associated with learning outcomes associated with serious games” Will help to defragment (organise) the literature and provide evidence for effectiveness of serious games. How to organise “learning outcomes” Cognitive, affective, motor, soft skills Education, corporate training (mostly qualitative?) SIGS
8. T6.2: UWS Literature Search terms and electronic databases Time period of search: 1961-Feb 2011 Search terms: ("computer games" OR "video games" OR "serious games" OR "simulation games" OR "games-based learning" OR "MMOG" OR "MMORPG" OR "M.U.D." OR "online games") AND (evaluation OR impacts OR outcomes OR effects OR learning OR education OR skills OR behaviour OR attitude OR engagement OR motivation OR affect) Databases searched:ACM (Association for Computing Machinery), ASSIA (Applied Social Sciences Index and Abstracts), BioMed Central, Cambridge Journals Online, ChildData, Index to Theses, Oxford University Press (journals), ScienceDirect, EBSCO (consisting of Psychology and Behavioral Science, PsycINFO, SocINDEX, Library, Information Science and Technology Abstracts, CINAHL), ERIC (Education Resources Information Center), IngentaConnect, Infotrac (Expanded Academic ASAP), Emerald and IEEE(Institute of Electrical and Electronics Engineers) Computer Science
9. Proposed year 2 activities for WP6.1 & WP6.2:Update the searchable database Search terms Should we include other search terms? If there are other search terms UWS would prefer to carry out a separate search so we can still compare the results of a new search with previous results.  Literature UWS will update the database using electronic databases to which we have access. Do partners want other electronic databases? Grey literature – volunteers? Ph. D. theses– volunteers?
10. Proposed year 2 activities for WP6.1 & WP6.2: Update the searchable database Languages All the papers we identified were published in English. But are we missing some published e. g. in Spanish (Baltasar and Pablo’s paper), French, German journals? How would we access these? Volunteers? Global reach We need to include the work of American researchers- Eva Baker, Sigmund Tobias and Dexter Fletcher.
11. Meta-analysis Proposed year 2 activities for WP6.1 & WP6.2: Meta-analysis Slight concern that there may be a relative absence of high quality RCTs. Should we do a systematic or narrative review too? The literature search would have to be done anyway. What quality of evidence do we accept in the meta-analysis? Do we focus only on RCTs? How about quasi experimental designs? surveys? qualitative studies? The search will identify many papers - from our previous experience over 7,000 between 2004-2009. It is a lot of work to search these for empirical evidence! Concern that we are doing work that was not budgeted for.
12. Inclusion criteria for selection of papers? Papers to include empirical evidence relating to the impacts and outcomes of playing games. RCTS – but dearth of these What time period are we searching? January 2004 to March 2012? Paper includes an abstract?
13. Classification of learning outcomes Do we lump all learning outcomes together or categorise according to GALA metrics: Cognitive (knowledge and skill acquisition) Affective Motor Soft skills Personal Interpersonal Are we interested in different learning outcomes across different sigs? Ages?
14. Rating the quality of the papers How appropriate is the research design? How appropriate are the methods & analysis? How generalisable are the findings of this study to the target population? How relevant is the particular focus of the study (including conceptual focus, context, sample and measures) for addressing the question or sub-questions of this review? To what extent can the study findings be trusted in answering the study question(s)?
15. Possible problem with the move away from surveys in WP6.1 WP6.1 originally specified that we should look at “stakeholder perspectives (e.g. parents, teachers, professors, trainers, educators) … highlighting the challenges that need to be overcome, the expectations that need to be addressed and the suggested research lines to facilitate adoption of SGs/SVWs in education settings” A possible problem with not doing surveys is that it might make it more difficult to make or retain contact with schools and educational institutions. Would we also want to carry out a literature review on attitudes to and acceptance and expectations of games to provide evidence about stakeholder challenges of introducing games into different educational sectors.
16. Proposed year 2 activities for WP6.1 & WP6.2: Meta-analysis We have already collected some evidence to form the basis for the meta-analysis: UWS carried out a literature review of learning outcomes up to Feb 2009. This paper was a narrative review which attempted to identify and categorise learning outcomes of playing entertainment and serious games. We need to update this to Feb 2012. We also have Tom Hainey’sanalysis of outcomes of games TUT provided a literature review on outcomes for playing games. UniGRAZ provided results of a literature search but this was mainly in the area of Neuroscience
17. Possible papers from year 1 activities on metrics? Can we get any papers from the year 1 metrics activities? 3 approaches Review of previous attempts to categorise learning outcomes of games Literature search of outcomes of educational games which have been reported in the literature Literature search for papers including serious games and neuroscience (Unigraz) Paper on metrics in serious games? Is someone else doing this?# Liz, Francesco, Michael Kickmeier, Rui Prada and Maria are currently writing a paper about engagement in entertainment games, but given the directive about entertainment games is this still a GALA paper? We will write a paper about engagement in serious games.
18. T6.3 Integration methodologies (15)Task leader: CNR-ITD, Rosa Bottino, MichelaOtt CNR-ITD and UNIGE– primary INESC-ID - secondary UCM - tertiary
19. Proposed year 2 activities for WP6.3 identify a selection of examples and best practices of the games which have been used successfully in educational contexts at the different curricular stages: nursery, primary, secondary and tertiary education. Some GALA partners have already been asked to identify suitable games /projects and we would ask all GALA partners to provide examples of suitable game use experiences. EU projectsin the field of games and learning (in particular those financed in the 6th and 7th framework program-ICT and in other relevant EU programmes such as the lifelong-learning program) will be examined.
20. Proposed year 2 activities for WP6.3 The reviewers suggested we use CHERMUG project Since WP3 reported minimal penetration of SGs in engineering and manufacturing, it might be worth exploring how SGs might be integrated into engineering education, especially at the secondary level where there is probably a reasonable chance for effective use. The panel recommends an effort be made to coordinate further exploration of how best to integrated SGs into education with WP3. Are reviewers suggesting we focus on only a few games?
21. WP6.3: Possible games/projects Primary School CNR: mind games Harri’s game – edu-elements – mathematics and chemistry – HarriKatemo The Code of Everand – Ian Dunwell, Coventry Uni, road safety game KristianKiili’sbrain training cell phones Consolarium in Scotland: Brain training: reasoning skills; Nintendogs: maths, resource management and communication skills 
22. WP6.3: Possible games/projects Secondary UCM: game to teach Resuscitation procedures 12-14 years; 334 students real world knowledge; paper to be published in English in Emergencies Me Tycoon (Kam) – extra- curricular Consolarium in Scotland: Guitar hero: in transfer from primary to secondary
23. WP6.3: Possible games/projects:Tertiary UWS: Requirements collection and analysis game for IT students; paper published UCM: – games for medical students, operating theatre, labwork ESADE:eFinance game teaching finance concepts to business students ESADE:Metavals – teaching statistics to students BIBA: Risk management and supply chain management for Masters students in engineering AAU: investment game for business success UNIGE Francesco – exploring entrepreneurship Games for digital electronics /computer programming UCM
24. EU Lifelong learning game projects Current CHERMUG: teaching research methods to nursing students, Liz Boyle STARTUP– secondary schools entrepreneur game, Thomas Connolly Goblin project – developing a game for language; Dr Hen van den Heuvel, CLST Radboud university, Netherlands Magical: Michela Ott Roman Breuer: RWTH Aachen – game to understand risk management in water supply/flooding Former ARG: Language game, Thomas Connolly - secondary ECLIL: Language game, Thomas Connolly - secondary
25. T6.4 Community of teachers and tutors (TU DELFT) Task leader: TU DELFT, Igor Mayer (3) UU, RemcoVeltkamp (3) HWU, Theo Lim (4) The community will emerge to some extent from GALA members, research and development contacts, stakeholders, association, outreach advisory board, ECGBL and other conferences The reviewers want us to: “explore link to existing communities and networks in the education sector investigating the phenomenon of serious games, e. g. those linked to existing LLP projects and those initiated by EUN (European Schoolnet)” There is a LLP network on serious games called SEGAN: Serious Games Network.
26. Presentation of D6.1: Year 1 time line
27. New 4 year time line
28. Self evaluation of progress Include a WP evaluation plan that includes measures of success in future annual reports and present progress against those measures at the next annual review meeting Metrics –one measures is the amount of work carried out, papers searched etc.