The Data Collection and Management Plan (The foundation of wargame design)
Wargame Data Requirements
The Data Collection and Management Plan (DCMP) is the plan that provides the foundation of the wargame’s design. The purpose of the DCMP is to lay out the roadmap that ensures all of the data required to initiate, conduct, and analyze the wargame is collected and processed effectively and efficiently. The creation of the DCMP begins once the problem definition process for the wargame, which is the process by which the sponsor’s objective and issues for the wargame are determined, scoped, and agreed upon by the wargaming team and the sponsor, is complete.
The most challenging aspect of DCMP creation is distilling the sponsor’s objective and issues into the information that the wargame’s analysts will require to produce the results that meet the sponsor’s requirements, the ‘analysis data.’ The DCMP also identifies the data that will be required by the wargame before the wargame’s start, the ‘initiation data.’ Finally, there will be data required during the wargame that provides the players with feedback on the results of decisions that they have made earlier in the wargame. This ‘feedback data’ is important to keep the players informed and engaged. It is likely that this feedback data will also make up some portion of the data that the analysts will need to do the post-game analysis and produce the results the sponsor is seeking.
Initiation Data: The data needed before the wargame begins will consist of the data that the players will need to understand their roles and make the decisions the wargame will require as the wargame begins and the data that any method, model, or tool (MMT) will need to be ready to function when called upon anytime during the wargame’s execution.
Example: A simple Red versus Blue wargame assessing Blue’s capability to defeat Red given a new suite of advanced sensors. Each side will need to know their Order of Battle (information about their forces, to include strength, disposition, mission, etc.) as well as weapon system capabilities (range, speed, etc.). They will also need to know the scenario, that is, the context of the battle (where, when, the road to war, etc.). Each side should also know some information about the other side, so some thought must be put into deciding how much information about each side’s opponent should be revealed to that side. In most cases, this information should match what information would realistically be available in an actual combat situation, given the collection assets that would be employed.
Let’s also assume we have an adjudication tool that will be able to decide the outcome of combat between the blue and red forces. The tool will likely need to be instantiated with each weapon system and sensor capability, location, and status. If a computer-based combat simulation is used as the tool, ensure that the time it will take for data to be produced and instantiated in the model is accommodated-many computer-based combat simulations require three to six months from the time data is requested until the time the simulation is ready to be used.
Feedback data: The data that is needed to keep the players engaged and to keep the wargame progressing is usually produced during the wargame. The challenge for the wargame design team is developing a means to produce and collect this data quickly and then communicating the appropriate data back to the players. Some thought must be given to the difference between what each player should know and what ground truth is. Battle Damage Assessment (BDA), a report of the damage inflicted upon the opposition, is often inaccurate and delayed. Should the wargame provide each player the ground truth or restrict the information to the player to only that information that the player’s assets could be reasonably expected to deliver on that timeline? What mechanisms and personnel (data collectors, analysts, controllers) will be needed to produce, collect, and disseminate the data? Is there a record of what data is provided back to each player if this data will also be needed for the post-game analysis? (If there is a significant time between the last round of player decisions and the feedback provided back to the players that allows them to understand the impact of their decisions, the wargaming team needs to think about what the players will be doing when the feedback data is being produced.) It is often the case that this feedback data is also a portion of the analysis data—the data that the wargaming analysts will need after the wargame concludes to process and provide wargaming results back to the sponsor.
Example continued: The first round of player decisions has concluded, and the wargaming white cell/controllers are determining the outcome of these decisions. Combat is being adjudicated by an adjudication tool. The players need to now be informed how their decisions played out in combat. What losses did their side take? What losses did they inflict on the opponent? Did their forces accomplish their mission and where are they now located? Because this wargame is assessing sensor capabilities, the wargaming team has ensured that the information being provided to each side is restricted to the information their sensors would actually produce. Because the information each player has available will be important to know when assessing the decisions that each player made during the wargame, the wargaming team also needs a record of the information each side has been provided as well as what ground truth, the actual status of each force, is at the conclusion of this first turn.
Analysis Data: This is the data that the wargaming team decided up front would be needed in order to produce the wargaming results that answer the sponsor’s issues and meet the sponsor’s objective for the wargame. This is the data that the wargame is designed to produce. This data may be collected or produced in many forms and by multiple sources. Data collectors using pen and paper may be transcribing player decisions while automated tools may be used to collect other information from players and/or wargame observers. Because the conduct of a wargame can often take longer than anticipated, wargames may have to conclude before all the information that is needed is produced within the wargame. A disciplined and well-organized wargaming team tracks the status of the required analysis data during the wargame, and makes in-stride adjustments to ensure the most critical analysis data is collected before the wargame terminates and players need to depart back to home station. The wargaming team may also be scrambling to conduct post-game interviews with key players from each side. Good wargaming teams realize that they will likely not have all the time needed to produce all of the required analysis data, and have developed contingency plans that provide alternate means to collect missing data. Wargaming teams can make good use of surveys and interviews as contingency data collection methods that enable the collection of missing data that had been expected to be produced from the wargame. Surveys and interviews can also be used as planned data collection methods. Surveys and interviews take time, which must be anticipated and planned for. Anticipate that surveys players are asked to complete at their home station may not get completed in a timely fashion, or even at all, so it is not a good idea to depend on remote surveys as a primary data collection tool. You may have surveys that players complete after each day of wargaming, or even between turns (perhaps when awaiting feedback data), but a careful balance must be struck that extracts as much useable information from each of the wargaming participants as possible without exhausting or frustrating the participants with hours of mind-numbing requirements that distract the players’ focus from their role in your wargame. Pre- and post-wargame surveys may be useful in assessing how the wargame may have changed participants’ perspective on the use of a new technology or Tactic, Technique, or Procedure (TTP).
Example concluded: The wargame has concluded after several turns. Each player may have planning sheets or forms designed for the wargame that need to be collected and delivered to the analysis group. Data collectors complete their forms or spreadsheets, and the quick look report is produced. The wargame participants are re-convened for the vetting of the quick look report to give the players a chance to correct the record if they were misunderstood, clarify results, and perhaps add a result that the wargaming team overlooked. Participants are informed of when to expect a coordinating draft of the wargame’s final report. Key players are then taken for their post-game interviews. Other participants are completing post-game surveys. The wargaming team collects all the data produced, and begins the task of detailed analysis.
Developing the DCMP
Developing a good, detailed DCMP is the key to designing a good wargame. Ensuring the wargaming team and the sponsor have zeroed in on the objective and issues that the wargame must answer is critical. This is problem definition. If the wargaming team fails to properly identify the problem, the wargaming results will prove to be of little or no use to the sponsor.
Analysis Data: Once the wargaming team has the problem (objective and issues) identified, then it must determine what data will be required to answer each issue. This is the analysis data. A framework for developing this data can be found in ABCA 354. This spreadsheet outlines a construct that distills each sponsor’s issue into Essential Elements of Analysis (EEAs). EEAs may be further decomposed into metrics. As each issue is dissected and its sub-components identified and documented, the spreadsheet is filled in. The completed spreadsheet will become the foundation upon which the wargame is designed. Once all the required analysis data is identified, the wargaming team will begin to understand the context, or scenario, that will be needed to provide players the opportunity to produce the data. The players that will be needed for that scenario will also begin to be informed from the completion of the spreadsheet. Which players need to be present to produce the required data, and what players must they interact with in the course of producing this data? The temporal aspect of the wargame will begin to emerge as well. Some issues may not be able to be addressed until others are resolved. There may be a series of vignettes, or mini-scenarios, that are needed to address all the issues, and where and when data will be produced may now be able to be identified. (The effectiveness of the Unmanned Underwater Vehicles at clearing mines will be assessed in the amphibious operations scenario’s lane-clearing vignette).
The emergence of the temporal aspect of the wargame will inform the type of wargame that will be required. Most good wargames are hybrid wargames that defy any attempt to label them as a ‘seminar’ or ‘system’ wargame, but it may be helpful to identify how the data will be produced. Data produced by direct player interaction is typically produced in a seminar environment and adjudicated by a facilitator. Data produced by indirect player interaction is typically produced in a turn-based system environment where a Method, Model, or Tool (MMT) provides the adjudication after assessing actions that were directed by each of the players. It is common to have wargames that will require data produced by both direct and indirect player interactions.
Feedback data: Most turn-based wargames will require some feedback mechanism so players can understand the impact of their decisions on the outcome of the previous turn’s adjudication. (In a seminar game, the facilitator typically provides both adjudication and feedback.) Some of the feedback data may also be analysis data, and may already accounted for in the spreadsheet. A careful analysis of the spreadsheet should reveal what feedback data is required and where and when in the wargame it should be produced. If the feedback data is not analysis data, then it may be useful to add this feedback data to the spreadsheet so all required feedback and analysis data requirements are identified in one document.
The wargame design can begin in earnest once all the analysis and feedback data have been identified. MMTs that are required to produce data can be identified (and perhaps annotated in the spreadsheet) and the task of procuring or developing these MMTs (and their data) can begin. Ensure that sufficient time is available if your MMT has significant data requirements or development timelines that make the MMT’s choice a risky one.
Initiation data: The last set of data to plan for is the initiation data. This is the data that will be required when the wargame starts. This data is the information that the players will require to prepare for their role and the first actions of the wargame as well as any data required by MMTs that will be used in the course of the wargame.
Some of the initiation data needs to be collected well before the wargame begins so the wargaming team can provide the appropriate ‘read ahead’ documents and/or briefings to the players. Players need to prepare for their roles, so they need to have the wargame’s scenario and the information about their specific roles at least a week ahead of time. This information is usually packaged and delivered to the players in the form of a professionally produced document often called the ‘read ahead packet.’ Players may have requests for information, or RFIs, once they have read through the packet and understand their role in the wargame.
MMTs that have significant data requirements may require substantial lead times to obtain, instantiate, and verify the data, especially computer-based combat simulations. If MMTs also have long run times, they may be run ahead of time if some range of input parameters can be anticipated from the players. MMTs can also be run during breaks for meals.
Completing the DCMP
Once a solid draft of the DCMP has been completed, the wargame design can begin. The data, scenarios and vignettes, MMTs, and players (the four elements of the wargame’s measurement space) are all informed by the DCMP. As the wargame design matures through several design-develop iterations, the DCMP should be revisited and updated as required.