A review of outcomes and impacts of playing computer games

1. A review of outcomes and impacts of playing computer games Dr Elizabeth Boyle University of the West of Scotland, High Street, Paisley, PA1 2BEliz.boyle@uws.ac.uk

2. Games for entertainment, learning and behaviour change • Over the last 40 years computer games have had a transformational impact on how we spend our leisure time and have increasingly replaced more traditional leisure activities • More recently there has been much interest in and speculation about whether games could be used for learning and changing attitudes and behaviour but less empirical evidence that games work

3. Literature reviews of games at UWS • At UWS we have been involved in carrying out a number of literature reviews on aspects of GBL • In this presentation I will describe a literature review which aimed to examine evidence very broadly about the outcomes and impacts of playing computer games • Focus was on positive impacts • Will highlight some interesting papers • Psychologist’s perspective!

4. Objectives of literature review • determine what research has been done on outcomes and impacts of playing computer games • interested in entertainment games, GBL and serious games • try to find a way of organising the diverse research in this area • describe how the different projects have defined and measured outcomes and impacts

5. Search terms ("computer games" OR "video games" OR "serious games" OR "simulation games" OR "games-based learning" OR "MMOG" OR "MMORPG" OR "MUD" OR "online games") • AND (evaluation OR impacts OR outcomes OR effects OR learning OR education OR skills OR behaviour OR attitude OR engagement OR motivation OR affect)

6. Databases searched databases relevant to education, information technology and social science. • ACM, • ASSIA: Applied Social Sciences Index and Abstracts, • BioMed Central, • Cambridge Journals Online, • ChildData, • Oxford University Press (journals), • ScienceDirect, • EBSCO (consisting of Psychology and Behavioural Science, PsycINFO, SocINDEX, Library, Information Science and Technology Abstracts, CINAHL), • ERIC, • IngentaConnect, • Infotrac (Expanded Academic ASAP) and • Emerald.

7. Selection of papers for inclusion in the review • Abstract: papers had to include an abstract. • Empirical focus: the paper had to report empirical evidence relating to the impacts and outcomes of playing games. • Date: papers from 2004 onwards were selected for inclusion in the current review: building on previous literature reviews • Age: use of games by adolescents over the age of 14 years

8. Categorising Games Games can be categorised in a number of ways: • Digital /non digital - Can non-digital games provide useful guidance about design of digital games? • The main aims/intentions of the game –entertainment, games for learning, attitude or behaviour change • Game genre (different kinds of game) • Platform/ delivery • Subject discipline or game domain

9. Game genre (Hertz, 1997) • action games (reaction based games including shooting and platforms) • adventure games (solving logical puzzles to progress though a virtual world) • fighting games • puzzle games (such as Tetris) • role-playing games • simulations • sports games and strategy games

10. Platform/delivery • computer games • video games • console games • mobile games • online games • MMOGs • ARGs • virtual worlds

11. Categorising the impacts and outcomes of games • Positive or negative effects • Intended or unintended effects • General or specific effects • Outcomes and impacts

12. Positive or negative effects • Negative effects: promote aggressive behaviour (Provenzo, 1991) and gender stereotyping (Bryce & Rutter, 2002); addictive (Griffiths & Hunt, 1998) • Positive effects: highly engaging; support perceptual and cognitive skills (Green and Bavelier, 2006); games support learning, behaviour change, social outcomes

13. Intended or unintended effects • Many of the outcomes of playing games are unintended, i. e. they were not intentionally designed into the game to achieve particular goals. • For example, violent games were not designed purposefully to make people more aggressive but they do seem to have this unintended consequence (Gentile and Gentile, 2007) • Entertainment games were not intentionally designed to improve visual perceptual skills but appear to do so • (Green and Bavelier, 2006). • Learning games are intentionally designed to support learning.

14. General or specific effects • Some papers examined the generic effects of playing games. For example several papers considered whether playing violent games generally makes players aggressive • Other papers examine the impact of playing specific games, e. g. emotional and physiological reactions to specific violent events in playing a game; does a game designed to support learning actually do so.

15. Frameworks for categorising outcomes of games • Garris, Ahlers and Driskell (2002) • O’Neill, Wainess and Baker (2005) • Wouters et al (2009)

16. Garris, Ahlers and Driskell (2002): learning outcomes • skills based learning outcomes • (technical and motor skills) • cognitive outcomes • declarative (knowledge of facts and data) • procedural (knowledge about how to perform a task) • strategic (the ability to apply rules and strategies) • affective outcomes (beliefs or attitudes)

17. O’Neill, Wainess and Baker (2005) CRESST model identified five “families of cognitive demands”: • content specific • content understanding • problem solving • content independent • collaboration / teamwork • communication • self-regulation

18. Wouters et al (2009) Identified four different kinds of outcome that games might have – • cognitive outcomes - knowledge and cognitive skills • motor skills • affective outcomes • communicative outcomes

19. Our categorisation of outcomes • knowledge acquisition/content understanding • perceptual and cognitive skills • affective, motivational and physiological outcomes • behaviour change • motor skills • soft skills/social outcomes

20. Diverse range of papers found Papers focusing on entertainment games (64) • Questionnaire studies/ development of models of time spent playing games/ game playing patterns/reasons for playing entertainment games • Validation studies of measures of engagement (flow, immersion) • Experimental studies of immersion in games, physiological responses • Experimental studies examining perceptual and cognitive benefits of playing games • Studies evaluating use of entertainment games for learning Papers focusing on games for learning (52) • Evaluations of the effectiveness of specific GBL in supporting learning • Studies examining pedagogical variables which influence effectiveness of games for learning Papers focusing on games for behaviour change (8) • evaluations of the effectiveness of specific games in supporting behaviour change

21. Affective and motivational outcomes • Most of the papers categorised under affective and motivational outcomes looked at aspects of motivation and engagement in playing games • Digital games are clearly highly engaging, but engagement is a somewhat elusive and underspecified construct • Engagement/enjoyment is an outcome of playing games but also a reason for playing

22. Indicators of engagement in games • numbers of people who play games • the amount of money spent on games • models of amount of time spent playing games/patterns of playing games (TAM model; addiction) • models of reasons for playing games (U&G theory) • the games that people play • subjective enjoyment of games - questionnaire measures • harder, more objective, measures of engagement

23. Models of time spent playing games • Reasons for playing games: U&G theory identified competition, challenge, social interaction, diversion, fantasy and arousal (Lucas and Sherry, 2004); applying U&G to sports games, Kim and Ross (2006) identified knowledge application, fantasy and identification with sport • TAM theory: social variables (social norms and critical mass) were more important than traditional TAM variables (perceived ease of use and perceived usefulness) in predicting time spent playing entertainment games (Hsu and Lu, 2004) • Self-regulatory mechanisms (relieving boredom, reducing loneliness, passing the time, providing an escape) are important in controlling amount of time spent playing games; the experience of flow make it more likely that players will have difficulties in regulating media consumption (Lee and Larose, 2007)

24. Studies of flow, immersion and presence Immediate feelings of enjoyment and engagement experienced while playing games • Flow relates to the activity • Presence relates to the technology • Immersion relates to involvement with game

25. EGameflow: Fu, Su and Yu (2005) • Sweetser & Wyeth’s model of GameFlow provided an integrated account of enjoyment of entertainment games. • Included eight dimensions: challenge, player skills, concentration, control, goal clarity, feedback, immersion and social interaction. • Fu, Su and Yu (2005) added a knowledge improvement scale to their EGameFLow model to measure “the level of enjoyment brought to the learner by e-learning games”.

26. Immersion (Jennett et al, 2008) • term used to describe the extreme involvement and enjoyable feelings experienced while playing computer games. • but lack of clarity concerning precisely what immersion means • series of experimental studies designed to develop better questionnaire measures of immersion (subjective measures) as well as exploring the possibility of developing harder, more objective measures of immersion and relate these two together.

27. Immersion Questionnaire (Jennett et al, 2008) A validation study identified 5 factors underlying the immersion experience questionnaire. These were 3 person related factors • cognitive involvement: “How much effort did you put into playing the game?” • real world dissociation: “To what extent was your sense of being in the game environment stronger than your sense of being in the real world? • emotional involvement: “To what extent did you feel emotionally attached to the game?” and 2 game related factors • challenge: “To what extent did you find the game challenging?” • control: “To what extent did you feel as though you were moving through the game according to your own will?”

28. Immersion (Jennett et al, 2008) • Predicted that participation in an immersive task would reduce the ability of players to subsequently “re-engage” with the real world and this could be measured by their performance on the subsequent task. • Time taken to complete the subsequent (Tangram) task was longer following the immersive game (Half-life) than following the non-immersive task (“square clicking”). • Interesting point: participants rated “square clicking” as quite immersive! • Participants’ eye movements significantly increased over time in a non-immersive condition but significantly decreased over time in an immersive condition. In an immersive game eye movements will decrease as players’ attention becomes more focused on visual components relevant to the game. • Also found more immersion with faster stimulus presentation.

29. Presence (Weibel, Wissmath, Habegger, Steiner and Groner, 2008) • Presence refers to a strong “feeling of being there", a feeling of actually being present in the virtual scenario that is being depicted in the game. • Weibel et al found that the three constructs, presence, flow (smooth and automatic running and adsorption) and enjoyment, were all strongly positively correlated while playing adventure game, Neverwinternights. Regression analysis confirmed that flow mediated the relationship between presence and enjoyment, i. e. presence leads to flow and that leads to enjoyment. • Participants who played against a human-controlled opponent reported stronger experiences of presence, flow and enjoyment than participants who played against computer-controlled opponents.

30. Are more technologically advanced and violent games more engaging? • Ivory and Kalyanaraman (2007) found that more recent, more technologically advanced games led to increased feelings of engagement and involvement (attention to the game, focus on the game, emotions experienced and physiological arousal which was measured by skin conductance and self-ratings of excitement). This finding would appear to justify the development of increasingly realistic games. • Ivory and Kalyanaraman also compared measures of presence and involvement for violent and non-violent games. They found no effect of violence indicating that the level of engagement was similar for violent and non violent games.

31. Role of violent content in player motivation and immersion • Przybyski, Ryan and Rigby (2009) predicted that enjoyment of, value of and immersion in games are predicted by players’ needs for competence, autonomy and relatedness. • They carried out two surveys and six experiments to examine whether violent game content further predicts player enjoyment, value and immersion in games. • Player ratings of enjoyment and value in games and desire for future play were strongly correlated with needs for competence and autonomy. • Violent content did not further predict enjoyment and value in games and desire for future play, nor was it related to player needs variables (competence and autonomy). • Violent content did not lead to more enjoyment compared with non violent games.

32. Engagement and learning • Many of the features that make games engaging also support effective learning. • Gentile and Gentile (2008): violent video games systematically and effectively use educational principles of learning, cognition, and instruction to teach aggressive behaviour.

33. Gentile and Gentile (2008): games incorporate 7 principles of effective learning (1) games should have clear objectives, at multiple levels of difficulty and should adapt to the prior knowledge, skills and pace of each learner (2) games require learning to be active with practice and feedback to the point of mastery. (3) knowledge and skills should be over-learned so that they become automatic (4) games should include both intrinsic and extrinsic motives. Motivational theory tends to emphasise the importance of intrinsic motivation to effective learning. (5) activities in video games should be presented in levels of increasing difficulty, complexity or pace, with skills at lower levels prerequisites for higher levels. This is especially true for technical and game skills. (6) video games encourage a close-to-optimal combination of massed and distributed practice (7) knowledge and skills should be learned and practiced in many different ways with several problems and examples across a variety of contexts

34. Online games promote higher level reasoning and argument skills • Steinkuehler and Duncan (2009) analysed the scientific reasoning skills displayed by players in their contributions to the online discussion boards while they played the popular online game, World of Warcraft (WoW). • They found qualitative evidence that players engage in high-level discussions and arguments in these fora such as using data and argument, building on others’ ideas and using system based reasoning. • Conclusion: games which were designed for entertainment can support higher level thinking and learning.

35. Impact of gaming on perceptual and cognitive skills • There is strong evidence that playing generic entertainment games leads to improvements in visual perceptual skills. • Green and Bavelier (2006) found that games players performed much more accurately on an enumeration task, where an array of squares is flashed up quickly on a screen. • There is a sharp drop in performance on enumeration tasks once a critical number of squares is reached (typically about four). The critical number is two more for VGPS than NVGPs. • VGPS were also better than NVGPs at tracking multiple objects simultaneously: game players could attend to two more items than non-games players.

36. Extent of the impact of games on perceptual and cognitive abilities • Boot, Kramer, Simons, Fabiani and Gratton (2008) compared performance of expert game players and non-game players on tests of attention,memory and executive control. • The cross sectional study showed that video games experts tracked objects moving at greater speeds, performed more accurately in a visual short-term memory test, switched between tasks more quickly, and made decisions about rotated objects more quickly and accurately. • A number of effects that had previously been found however were not significantly different in the current study for gamers versus non-gamers. These included performance on an attentional blink task, enumeration, short term memory, mental rotation, o-span, tower of London and Raven’s matrices tasks.

37. Extent of the impact of games on perceptual and cognitive abilities • Barlett et al (2009) examined whether playing video games helps players to cognitively ‘‘warm-up.” They found that playing (violent or non-violent) video games between two trials of a cognitive test (assessing short term working memory, visual attention, mathematical decision making and auditory perception) led to improved performance relative to those in the control group.

38. Games for learning: knowledge acquisition/content understanding evidence of increases in knowledge acquisition and/or enjoyment in games across curricular areas • structures in Civil engineering (Ebner and Holzinger, 2007) • vocabulary learning (Yip and Kwan, 2006) • the operation of the heart (Cameron and Dwyer, 2005) • computer memory concepts (Papastergiou, 2009) • the respiratory chain of the inner mitochondrial membrane (Yaman, Nerdel and Bayrhube, 2008) • statistics (Nte and Stephens, 2008) Many learning games are puzzle or quiz type games which arguably do not use the full benefits of game based learning.

39. Pedagogical variables influencing effectiveness of GBL Several authors make the point that simply introducing a game into learning will not necessarily lead to better learning. • those with prior videogaming experience consistently performed better than those without (Orvis et al, 2008) • providing narrative structure in an ARG led to increase in knowledge but not to a more positive attitude to collaborative learning nor more motivation for learning history (Huizenga, Admiraal, Akkerman and ten Dam, 2007) • feedback provided to players was important in effective learning (Cameron and Dwyer, 2005) • worked out examples had a more positive influence on the learner’s situational subject interest than problem solving tasks (Yaman et al, 2008)

40. Implicit learning • Ciavarro, Dobson and Goodman (2008) incorporated an implicit learning mechanism into a computer game to play ice hockey • by penalizing aggressive behaviour or rewarding positive behaviour (with a greater chance or scoring and winning) the “karma” mechanism aimed to change players’ behaviours to be less aggressive, although players were not explicitly informed of this goal in the general instructions provided about how to play the game. • implicitly rewarding good behaviour could lead to positive changes in game-play behaviour.

41. Games as a new tool /methodology in studying psychological characteristics • van Reekum and Johnstone (2004) manipulated goal conduciveness (whether the event was congruent with the desired goal or not) and intrinsic pleasantness (subjective appraisals of emotional events) on physiological responses (cardiac activity, skin conductance, skin temperature and muscle activity and emotion self-report) while playing an action adventure game (Xquest). • Goal conduciveness was associated with significant autonomic effects including changes to interbeat interval, pulse transit time, skin conductance and finger temperature while the manipulation of intrinsic pleasantness had little impact on physiological responses. • van Reekum and Johnstone view the computer game paradigm as an important new tool or methodology to examine variables in emotion research.

42. Conclusions • Playing computer games leads to diverse outcomes and impacts • Useful to develop an organising framework to describe outcomes and impacts • Some of the strongest evidence of the impact of games is in the area of “unintentional” outcomes • Many of the features which make games engaging also support effective learning

43. Areas for future research • Further exploration of links between engagement and learning • Examine what different game genre can offer in more detail • Explore use of computer games in social science • Examine use of computer games as a tool or methodology in psychological research