620 likes | 815 Views
The Science of Play Testing: EA’s Methods for User Research. The Science of Play Testing: EA’s Methods for User Research. Veronica Zammitto Game User Researcher. Outline:. Game User Experience Evaluations Case Study 1: NBA Live 10 Case Study 2: NHL 11 Take Away Q&A.
E N D
The Science of Play Testing: EA’s Methods for User Research The Science of Play Testing: EA’s Methods for User Research Veronica Zammitto Game User Researcher
Outline: • Game User Experience Evaluations • Case Study 1: NBA Live 10 • Case Study 2: NHL 11 • Take Away • Q&A
Mixed Method Approach for Evaluating Sports Games • In-depth understanding of User Experience (UX). • Support design decisions. • Triangulating: Eye Tracking Interviews Survey UX Telemetry Psychophysiology
Case Study 1: NBA Live10 • NBA gameplay issues to identify: • successful and unsuccessful gameplay aspects. • emotional profile of the player • engagement and emotions • attentional focus • UX throughout the game, and for certain events.
Eye Tracking • Hardware + software X, Y on screen • Tracking users’ gaze can reveal the player’s focus. • DIYS, ~US$ 4,000 to 80,000
Eye Tracking • By using ET we can identify where players’ attention is. • Fixation • Saccades • Gaze Movement • Patterns
Lay Up Dunk Pass Call Time Out Switch Players Steal Telemetry • Hooks in the game engine that flag and time stamp pre-defined events. • Players’ in-game behavior • Statistical analysis • Visualizations • Machine learning algorithms
Players’ Scoring Location 58.6 % 91.4 % of the shots
AI Scoring Location 75 %
What Went Right – NBA study • Better understanding of : • players styles and demographics. • In-game behavior • Identification of emotions (next slides) • The new techniques were proven to provide useful data to development. • Rethink the role of game elements. I.e., coach. • Create tutorials for court observation based on eye tracking data • Worthy of further investment to continue with studies
What Went Wrong • Large scope of the NBA study • Impacted synchronization of the usability study with production’s delivery schedule. • The study should have been subdivided into mini assessments to achieve a quicker turn around. • Low involvement of production in the project. • Manual coding: • Time consuming.
Case Study 2: NHL 11 • Same techniques used for NBA • Adjustments from lessons learnt: • Narrower focus: “Game Presentation” • Front-End Visualizations (Overlays). • I.e.: Do players look at information provided in the UI? • Cut Scenes (NIS): • Are they watched or skipped? • High involvement with the development team • Meetings with development for a ‘statement of work’. • Iterative process with development • Helps to define the root of usability questions.
Only ¼ of the overlays are actually observed, the other 75% are ignored.
Percentage of Observed and Ignored Overlays during different game sections. • Quality and sensitive information that helps the player has more chances to be looked during NISes.
Overlay “Map” in NHL 11with Observed and Ignored proportions
Non-Interactive Sequence (NIS) Event NISes Scripts - Subsets
The effectiveness of NIS is a combination of its type (Canned, Context Sensitive, and Replay), the event that triggers the sequence, and its frequency.
Psychophysiology (Biometrics) arousal • Infer emotions from physiological data. • Arousal: engagement, excitement, magnitude of emotions. • Valence: positive (fun) or negative (frustration) Frustrating Exciting Emotional Valence Boring Fun
Galvanic Skin Response (GSR) • Psychological arousal. • Skin’s conductance increases when a person becomes excited, stressed, or anxious.
Electromyography (EMG) • Emotional valence. • Sensors capture and amplify muscles’ contractions. • Facial muscles: • Smiling (zygomatic) = Positive emotions • Frowning (corrugator) = Negative emotions (or) Cognitive load
Player’s positive emotional reaction when scoring in NBA Live 10
Emotional Profiling of NHL 11 Arousal = exciting, engagement.