Wargaming Education Seminar: An Overview of the Game Project Management Process

1. Wargaming Education Seminar: An Overview of the Game Project Management Process

2. War gaming project flow 100% 80% FPC 20% 40% 70% Tasking Proposal SponsorApproval Identify Purpose Design Objectives Design doc Alpha test Development Playability Process Products Tech Integrate DCAP Testing Tool Load DCAP Tools Beta Rehearsal Game Mechanics Presentations Plenary practice Execution Game Play Tool Data Surveys Plenary Analysis Data Aggregation Coding Game Report Executive briefing MPC Post-Game War Game IPC CDC 1-2 mo. 1 mo. 2 ½ mo. 1 week 2 mo. 2 ½ mo. 1 mo. • War Games may take as little as 2 months to develop and execute or as long as 10 - 12 months depending on scope, complexity and sponsor requirements

3. Game Project Management Process

4. Tasking Phase • “My staff isn’t big enough. Can you guys do a war game on that?” • “I don’t enough ships to do what I’m supposed to. Can you guys do a war game on that?” • “I need different kinds of ships to do what I am ordered to do. The ones I have do not permit me to accomplish my tasks.” • “I want to train my subordinate commanders to fight with poor communications. Can you guys do a war game on that?” • “My sailors are not paid enough. Can you guys do a war game on that?” Input: Game request letter received by NWC from prospective sponsor Output: Game proposal doc from WGD to sponsor w/ sponsor agreement confirmed at Initial Planning Conference (IPC) 1-2 months

5. Coordination with game sponsor • Sponsor- person (admiral or general) from an organization who requests a war game

6. Problem defined problem nounprob·lem \ˈprä-bləm, -bəm, -ˌblem\ 1. a: a question raised for inquiry, consideration, or solution b: a proposition in mathematics or physics stating something to be done 2. a: an intricate unsettled question b: a source of perplexity, distress, or vexation http://www.merriam-webster.com/dictionary/problem

7. Problem definition • “If were given one hour to save the planet, I would spend 59 minutes defining the problem and one minute resolving it.” Albert Einstein

8. Distinction among problems… OPS RESEARCH (M&S) BASIC RESEARCH GAMING

9. Different kinds of problems • Manpower study • What is the minimal manning for a submarine to accomplish all currently assigned tasks? (Not a war game!) • Engineering study • What’s the optimal thickness of ship skin to absorb missile strike and permit damage control team time to respond to casualty? (Not a war game!) • Force employment options • How should missile defense ships be positioned and protected? (yes, a war game)

10. What is the Nature of the Problem? • Urgency? • Near, mid, long-term? • Criticality? • People dying? (NOW!) • Bureaucratic procedural improvements? (long-term?) • What’s broken? • What’s wrong with how things are done now? • How should the problem be framed • What is the scope of the problem to be explored?

12. Problem statement • A problem statement is a clear concise description of the issue(s) that need(s) to be addressed by a problem solving team. • It is used to center and focus the team at the beginning, keep the team on track during the effort, and is used to validate that the effort delivered an outcome that solves the problem statement. How To Write A Problem Statement | Ceptara www.ceptara.com/blog/how-to-write-problem-statementMar 18, 2009

13. Defining the problem 1. Explore the current situation. Paint a picture in words by including the “presenting problem,” the impact it is having, the consequences of not solving the problem, and the emotions the problem is creating for those involved. 2. Explain. Once you have examined and clearly explained the situation, draft a simple problem statement by filling in the blank: The problem that we are trying to solve is: ___________. Distill the problem to its simplest form possible. 3. Ask yourself. “Why is that a problem?” If the answer is another problem, then congratulate yourself for moving from the “presenting problem” to a deeper problem. Then ask yourself again, “Why is that a problem?” Do that repeatedly until you either land on what is obviously the source of all of the problems you’ve identified or you identify unexpected consequences of not solving the problem. If you land on unexpected consequences, the problem you identified right before that is likely your “source problem.” Toyota famously created the "five why’s'' technique for their Six Sigma process improvement program. While that number was limited to five why’s, the truth is sometimes it takes only one why. Other times, it may take 17. Ask as many times as needed until you get to the source problem. Also, ask if it is a problem worth solving -- many problems aren’t. http://www.entrepreneur.com/article/237668

14. Writing a problem statement Who - Who does the problem affect? Specific groups, organizations, customers, etc. What- What are the boundaries of the problem, e.g. organizational, work flow, geographic, customer, segments, etc. - What is the issue? - What is the impact of the issue? - What impact is the issue causing? - What will happen when it is fixed? - What would happen if we didn’t solve the problem? When- When does the issue occur? - When does it need to be fixed? Where - Where is the issue occurring? Only in certain locations, processes, products, etc. Why- Why is it important that we fix the problem? - What impact does it have on the business or customer? - What impact does it have on all stakeholders, e.g. employees, suppliers, customers, shareholders, etc. Each of the answers will help to zero in on the specific issue(s) and frame the Issue Statement. Your problem statement should be solveable. That is, it should take a reasonable amount of time to formulate, try and deploy a potential solution. http://www.ceptara.com/blog/how-to-write-problem-statement

15. Game Purpose? • Educational • Goal is learning • Example: practice using the operational planning process • Experiential • Goal is practice • Example: practice for a staff working with a new war plan • Analytic • Goal is research • Example: what is the preferred multinational command and control organizational structure for anticipated operations?

16. Tasking PhaseSummary • Often unofficial query, often by email or phone; Chair briefed by receiver • Informal liaison between NWC WGD and prospective Sponsor to understand potential sponsor’s problem and if war gaming is an appropriate technique to address the sponsor’s problem • WGD chair designates someone to conduct liaison (often prospective game director, depending on work load) • With WGD Chair approval game team formed • Concept Development Conference (CDC) explores sponsor problem in more detail • Sponsor, WGD discussions codified in a game proposal doc, agreed to at Initial Planning Conference (IPC) Input: Game request letter received by NWC from prospective sponsor Output: Game proposal doc from WGD to sponsor w/ sponsor agreement confirmed at Initial Planning Conference (IPC) 1-2 months

17. Design phase • Game proposal document codifies mutual agreement as discussed at the Initial Planning Conference (IPC) • Game team, led in this phase by game designer, works to figure out how to get at sponsor problem, informed primarily by the game’s objective as characterized by the analyst w/ team collaboration. All hands contribute to help designer. • Initial adjudication plan • Initial game process review and products assessment needed for analysis to attain game objectives • MPC , design and initial manning doc agreed to (done!, slap table, move on!) Input: Game Proposal Doc, Initial Planning Conference (IPC) Output: Game Design Document, Mid Planning Conference (MPC) 2 1/2- 3 months

18. Many opportunities for distraction as a designer! (how many degrees rolling right?)

19. But there are some basic design things to do

21. Development phase • Refine the game’s blueprint (game design doc) to ensure game “playabilty” • Make sure all the individual piece parts work as expected • Develop game process, products, and tech • Process: • inputs to players • player activities • player outputs • data collection for analysis • refine time estimates for how long things take • Products: • briefings • templates • Tech • Web, network, AV Input: Game design document Output: Game interfaces complete and functional 2 ½ months

22. Designer vs. Design Phase Distinction “Design”- the phase (time) “Designer”-the person Design doc Write design doc, ideally having been informed during initial discussions with the potential game sponsor Brief design doc at MPC Facilitator Guide Write Fac Guide (initial version) provided to designated facilitator for subsequent refining by designated facilitator) • Follows after Tasking Phase • Ends at MPC w/ Sponsor agreement w/ design doc • But, other WGD roles performed during “Design Phase” • Director: project schedule management • Analyst: lit rev, RQs, initial survey questions tested at Alpha • Log: MLH facility capacity • Security: classification • KM: initial liaison w/ GT, Web

23. Development The Development Phase: “Game development seeks to ensure that the game design is complete, as realistic as possible or desirable, and that it is playable and capable of meeting the objectives specified for it” - Peter Perla The Game Developer: The person responsible for game playability. This person molds the game designer’s ideas into a refined process for the participants to be able to play the game. - NWC War Gamers’ Handbook

24. Playability • Players will need to understand how the game works • What they will be expected to do/how • Information presented to them • How decisions they make become an input to the game • How player inputs affect the game • Player Motivation • Intrinsic/Extrinsic rewards • Producing enjoyment - Fun, Exciting, Rewarding (winning), etc. • Boredom - Lengthy surveys, long days, etc. • Player experience and use of technology/player interface “It just works” - Steve Jobs

25. Transition from Design to Development Resources Development Time Complexity Design Time

26. The Design Development considerations provided by the designer • Purpose and game objectives • Type of Game • Analytic, educational, experiential • Inductive or Deductive • Level of war • Number of sides: One, two, multi-sided • Players • Role (Established entity or notional) • Expertise, numbers, rank, etc. • Nationality (security, classification guidance) • Classification/player nationalities • Physical Security • Availability of spaces • Time: Move-Step vs. Running Time • Timeframe (current capabilities or future) • Type of adversary (fictional or specific country “like”) • Order of Battle • Scenario • COP/geographic visualization requirements • Location (home or away) • Tools available (computers, paper, MTMU, etc.) • Type of adjudication • Player activities • Player Inputs/Outputs • Activity • Communication* • Survey instruments/facilitated discussion

27. Use of TechnologyGeneral Principles User Interface Data Capture Data capture should be unobtrusive Drive toward capturing during game play vs. surveys Keep players in role Data captured in a usable format Structured Easy to export/dynamically accessed Recognized format for analytic tools Meta Data Tagging • Technology should work, be easy to understand and Intuitive • Players usually do know • MS Office (Word, PowerPoint, Outlook, etc.) • Web browser, Windows Explorer, Adobe Reader • Mouse/keyboard, basic office hardware • Players usually don’t know • Custom applications/portal • Cutting edge technology (e.g. google glass) • Specialized applications (CAD, GIS, etc.)

28. Basic Information Flow Adjudication & Analysis Information Storage & Processing Input Output

29. Game Project Management Process

30. Testing phase • Integrated testing of people, process, and tech • Use simulated players not part of core game team to expose previously overlooked problems • Iterative as required Input: Game products and interfaces complete and functional Output: Successful beta test and finalized documents; final planning conference (FPC) 1 month

31. Rehearsal phase • Practice individual duties, roles in game • Briefing deliveries • Facilitators • Practice welcoming players, introducing computer and log ins, accessing player products, explaining cell schedule and process • Refine, personalize, take ownership of facilitator guide Input: Successful beta test, FPC Output: game team prepared to perform game duties 1 month

32. Execution • Welcome, orient players (PNWC involvement?) • Support cell facilitators as needed • Execute social, dining plans • Consider special requirements if flags present • Manage game information flow (adjudication, scenario updates, data collection) • Daily game team meetings as required (such as pre-game game, lunchtime, and post-game day huddles) • Game director coordinate w/ sponsor Input: Players arrive, game team prepared Output: Players depart, game data collected 1-2 weeks

33. What’s War Game?

36. Analysis and Archiving Phase • Lead analyst manages data collection and analysis team (DCAT) to conduct analysis • WGD Chair provided periodic progress updates • Post-game executive brief presented to WGD Chair, possibly provost, PNWC • Game report archived at: T:/War Gaming/GAME EVENTS/War game name • Game products archived at: T:/War Gaming/PRODUCTS ANALYSIS Input: Player generated data Output: Game report, game executive briefing, report archived 2 months ARCHIVE

38. Game Project Management Process

40. UNCLASSIFIED Joint Staff Cyberspace Operations C2 War Game Problem There is a lack of understanding of how, when, or if combatant commanders should transition from a Direct Support model to an OPCON model for the C2 of cyber forces, as directed by CJCS the EXORD. Purpose Examinedifferent cyber C2 models to inform a 2015 Joint Staff Operations Deputies (OPSDEPS) decision on the rationale of transitioning to the OPCON C2 model. Objective Examine thestrengths and weaknessesof alternative C2 organizational structuresfor cyber forces in order to foster greater understanding of cyber C2. UNCLASSIFIED

41. C2 literature reviewed

42. UNCLASSIFIED RQ 1: Which of the three models, if any, for Operational Cyber C2 presents the greatest number of strengths relative to its weaknesses? RQ2: Which of the three models for Operational Cyber C2, if any, presents the greatest number of weaknesses relative to its strengths? RQ3: What are the primary strengths and weaknesses of each model? Null Hypothesis: Of the three models of operational cyberspace command and control presented in this game, no one model exhibits any greater degree of strengths or weaknesses than any other model. Affirmative Hypothesis: Of the three models of operational cyberspace command and control presented in this game, there is model exhibits any greater degree of strengths or weaknesses than any other model Research Questions and Hypotheses UNCLASSIFIED

43. Design elements • Analytic • Deductive • Testing to the null hypothesis: no one C2 model is preferred relative to C2 criteria used in this game • 1 ½-sided • Operational Level of War • 6 player cells (GCC + USCC cell/ C2 model); White Cell (RFI response, C2 review panel) • 75 players (expertise from CCDR JCC’s, USCC, JFHQ-C, JFHQ-DoDIN, JFHQ-NMT, Service Components, FCCs, plus White Cell SME’s) • Year of Game: 20XX • Scenario: 2 separate vignettes, informed by Defense Planning Scenarios and recent cyberspace activities • Game time: separate vignettes (vignette 1 results not input to vignette 2)

44. Design- cells White Cell (other GCCs, TRANSCOM, SVCs, etc.) General Support OPCON TACON GCC JCC USCC GCC JCC USCC GCC JCC USCC • Player inputs (all get same): • Vignette 1 or 2 • HHQ tasking • Planning guidance • Forces GCC and USCC brief in turn their integrated CO plan to panel of 0-6’s before it goes to respective CCDR’s CUB GCC and USCC brief in turn their integrated CO plan to panel of 0-6’s before it goes to respective CCDR’s CUB GCC and USCC brief in turn their integrated CO plan to panel of 0-6’s before it goes to respective CCDR’s CUB C2 SME C2 SME C2 SME DODIN SME DODIN SME OCO SME DCO SME OCO SME DCO SME OCO SME DCO SME DODIN SME

45. Design: Player Activities Overview Problem identification (“MA”) Develop method to address, solve problem (“COA Dev”) Consider tools available (OOB, capability shortfalls?) GCC & USCC coordinate capabilities, actions “pre-brief” SME panel on proposed CO plan, including risk & adequacy of C2 SME’s critique, Q&A w/ teams on rationale, problems w/ plan and C2

46. Game execution schedule 9-13 Mar 2015

47. Analytic approach • Deductive • Qualitative • ACH (analysis of competing hypotheses) • Hypothesis: no one model better than another

48. 1. Based on a C2 organizational structure for cyber forces and its efforts to accomplish assigned missions, which of the 7 cyber C2 attributes provided represent this C2 structure’s greatest strengths? Examine the strengths and weaknesses of alternative C2 organizational structures for cyber forces in order to foster greater understanding of cyber C2 Objectives Data Collection & Analysis Plan Framing 2. Based on a C2 organizational structure for cyber forces and its efforts to accomplish assigned missions, which of the 7 cyber C2 attributes provided represents this C2 structure’s greatest weaknesses? Research Questions & Hypotheses H0: There is no preferred C2 model that allows for greater efficacy in accomplishing mission based on the 7 cyber C2 attributes • Note: Subsidiary research questions 1 through 6 focus on each of the above mentioned strengths and weaknesses of the 3 C2 models by applying these 7 attributes to the following activities: • Cyber ISR • OCO • DCO-IDM • DCO-RA • DODIN OPS Ha: There is a C2 model that allows for greater efficacy in accomplishing mission based on the 7 cyber C2 attributes • Deductive • Qualitative • Analysis of Competing Hypotheses

49. Datasets and Tools for Post-Game Analysis

50. C2 Structure(Independent) • Strength or Weakness of C2 Model assessed on each of 7 Attributes (Dependent) • Scenario/Assigned Mission(Moderator) • Scenario/Assigned Mission(Moderator) • Cyber ISR/OCO/DCO-IDM/DCO-RA/DODIN OPS(Moderator) Independent (X)= Variable manipulated by the researcher Dependent (Y)= Variable resultant from interaction with the independent Moderator (Z)= Variable that affects the direction and/or strength of the relation between an independent and dependent variable Identification of Research Variables