Wargames Illuminate Cyber Threat Discovery

Wargames illuminate cyber threat

Classroom seminar games provide a powerful analytic and educational toolset

By Olivia Kay Hernandez, Theodore T. Allen and Douglas A. Samuelson

Wargames produce innovative thinking, evaluation of information and the development of strategies. Image © Elena Duvernay | 123rf.com Image

We developed two in-classroom seminar wargames to analyze and teach the prospective effects of proposed courses of action in response to cyberattacks. These were part of a research effort for the Army Cyber Command and Second Army (ARCYBER / 2A), under an NSF grant. These quick and simple wargames illustrate well the advantages such games can provide.

Wargaming is making a comeback. Wargames produce innovative thinking, evaluation of information and the development of strategies [2], [6], [9], [12], [14], [18]. Because of this, two years ago Deputy Secretary of Defense Robert Work strongly urged Department of Defense components to use more wargaming in analysis [17]. From seminar games to advanced computer simulations and large-scale, highly structured exercises, wargaming is a vital tool that is proving applicable to myriad scenarios.

Playing wargames has many benefits, for both the participants and those who create the games. The players can develop new thought processes, be exposed to original ideas and concepts, and put their unique skills and expertise to work. Game creators can tailor the games to meet their specific needs and goals. Goals can range from specific, such as determining whether a battalion is robust enough to defeat an enemy in a specified scenario, to the very general, such as examining new techniques or innovative policies. Wargames can include forms of competition and assertiveness other than, or even instead of, direct military conflict (Cf. [13]). Wargames can teach players about many different aspects of a conflict. The back and forth of receiving information and making decisions is reflective of everyday life. Further, dealing with actuality and others with differing objectives allows wargames to produce realistic outcomes. In general, wargaming is both an organized and creative method from which all involved can benefit.

The setup of the wargame needs to reflect the research question and appropriate scope and level of detail. For our project, we created two educational seminar-type cyber threat discovery games. Wargames played for discovery can have multiple goals, including “the devising, executing and testing of courses of action against an enemy, in order to explore some military problem or proposed future situation” [14]. These games have less rigor to allow exploration and novel outcomes. Seminar-type games produce more qualitative findings, and generalized concepts have been determined and are reviewed in the results section of the games’ after action report [11].

The goal of these wargames was primarily to create educational and enjoyable games that expose brigade-level personnel to cyber threats and explore how to build organizational resilience. These approximately one-hour long games build teamwork among players by providing awareness of how interconnected a cyber security breach can be. They also provide benefits by allowing the players to understand and relate to cyber experts, and assist in dispelling the concept that cyber security knowledge is limited to certain fields or disciplines.

Let the Games Begin

The games are based on real-world events in the news (not directly related to military incidents, although the implications are clear enough to be informative) [4], [5], [7], [8], [15], [16]. The first game is called “Slapdash vs. LateSpunk.” In this game, two different commercial airlines are competing for military contract business. A lieutenant colonel (LTC) must get troops to both New York and Afghanistan. The reasons for the travel are classified. The airline that the military currently uses (Slapdash) has experienced a cyber security breach. That same airline has also had various issues in the past. A different airline (LateSpunk) might be able to sway the military commander to its side and capture new business.

Figure 1: Sample of information provided to wargame players.
You are in charge of monitoring breaches at U.S. military institutions of higher learning. You participate in system audits and training exercises. You are aware that viruses and limited breaches occur with regularity, but true sensitive breaches are considered rare.
P - Wing
You are in charge of students at a military academy conducting graduate research with strategic partners. Almost all of the work is unclassified and on a computer system. Several contractors work with systems under your purview, and there might be guidelines for handling sensitive information that you and they do not know.
You run several portals that students and partners use. The portal identity credentials are shared with the employment cac (“passport”) identifiers, and there is some sensitive information on the portals. You have recommended multifactor authentication, but it has not yet been implemented due to time and expense.
P - Cyber
Security Expert
You train students and assist with offensive and defensive cyber security operations. You are not aware of classified data on a local portal but are aware that some base systems are not up to DOD standards and might be required to reach those standards.

Players must work to assess business vulnerabilities, determine priorities and analyze the circumstances. Leadership and various departments come together to investigate both the nature and scope of the breach, and to agree upon the most appropriate response. Ransomware is also involved, and the players must decide upon the most suitable action.

“Portal Breach” is the second game. A military university has experienced a cyberattack. There are both inside and outside users of the portal, and they might have contributed to the breach. The goal of the attack is not known, but action must be taken to protect the portal.

One player is an external hacker, and the other players are all involved in determining how to manage the breach. “Portal Breach” helps players to understand how distrust and possible humiliation can occur in a cyber security investigation. It also brings to light time sensitivity concerns. Further, the players are attempting to determine the hacker’s incentives and limits.

Both games follow the same format and rules. Phase Zero is preparation. First, the participants are assigned their roles (airline CEO, military cyber expert, etc.). Based on the assignments, the players are given specific information, such as an expert who performs defensive cyber activities and forensic investigations.

After the initial information is provided to establish the roles, all players receive more tailored information during Phase One. Examples include knowledge that the core server has been infected or that the FBI has contacted you regarding IP addresses for your portal. Players are then free to deliberate for approximately 17 minutes. All participants are free to speak to others or break out into separate groups.

Certain players, designated as the principals, provide statements after the deliberations have concluded. Each principal is given around three minutes to speak. While other players may have rebuttals or questions during this time, the focus is on the principals’ statements. Following the statements there is a die roll to choose one of the possible actualities of the cyber security incident: Perhaps the system backup is easily restored with no data loss, or maybe inside and outside users were involved with the breach. The outcome and facts selected via the die roll are shared with all participants.

Phase Two begins with additional personalized information provided to the players. Again, players have time (17 minutes) to deliberate and discuss the newly shared information as well as respond to the ground truth. Afterwards, the principals provide their closing statements (three minutes each) to the group. The result is a consensus of the outcome, such as the military deciding which airline they will hire. Figure 1 offers an illustration of the information given to participants. The information conveniently fits onto standard 8.5” by 11” sheets of paper.

Findings from the Games

Figure 2: Slapdash vs. LateSpunk topic analysis.

Slapdash vs. LateSpunk provided many thought-provoking results. First, the CEO of LateSpunk did not use the breach of Slapdash effectively in order to gain the military’s business. The CEO did not get a commitment from the LTC that LateSpunk would be its airline going forward. Also, the fear of not having sufficient information to make decisions greatly hindered LateSpunk’s CEO.

When the actuality was shared with the group, which in this case was a $1 million breach, the ability of Slapdash to bargain with the LTC became quite limited. However, the CEO of Slapdash missed an opportunity to “muddy the waters.” Slapdash could have used findings from its own cyber experts about the severity and impact of the incident.

Portal Breach had different outcomes from Slapdash vs. LateSpunk, as all players were essentially working together to minimize the hacker’s impact. The wing commander had ultimate authority to shut down the portal. However, there was a lack of organization and leadership to quickly confirm if this was necessary. Due to passively waiting for additional discussion, the opportunity was lost to thwart the hacker by shutting down the portal and preventing additional losses.

The hacker was very careful initially to prevent being discovered. However, the hacker’s presence was already known by the portal admin and cyber expert. Near the end of the game the hacker realized this and determined that being cautious no longer provided any benefit.

Figure 3: Portal Breach topic analysis.

Another interesting result from Portal Breach involved the actions of the inside and outside users. Even though (due to the die roll) they both contributed to the breach, they were very adamant that the portal not be shut down. The inside and outside users both required information from the portal to complete their tasks and pushed to keep the portal accessible even at the risk of the hacker gaining additional data.

We used two rapporteurs for these games. They took ample notes based on the conversations of the participants, including their concerns and reasoning for their decisions. We then processed the notes through a text analysis program to create topic groupings. Figure 2 shows the top 10 topics from Slapdash vs. LateSpunk, while Figure 3 shows the same information for Portal Breach. Stemming using the Porter algorithm trims words so that the plural or other forms/tenses of the same word can be appropriately clustered using Latent Dirichlet Allocation and directed clustering [1], [3], [10].

Summary and Conclusions

Based on feedback from the participants, we met the goal of creating enjoyable games that provide a realistic presentation of cyber security breaches. A controversial critique of the games was that not enough information was provided to the participants in order to make fully vetted decisions. However, other players stated that the lack of information was very realistic and an accurate representation of cyber security incidents. This observation can be extended to decision-making at large.

We presented our preliminary report and assessment of these games at the Military Operations Research Society (MORS) Symposium in June. Our audience, including a number of noted experts in cybersecurity and wargaming, offered various suggestions for improvement but found the games credible and beneficial.

If others are planning to play both games, we recommended to begin with Slapdash vs. LateSpunk. The starting point of this game was more obvious to the players as there were two clearly defined teams working with the military. Portal Breach allows for more collaboration between different organizational levels. This game can foster substantial teamwork or harmful distrust, which allows it to showcase the interplay of differing personalities and viewpoints.

Overall, our study indicates that these games are useful tools in exposing players to cyber security incidents. Future enhancements to the games involve incorporating feedback and more advanced cyber security concepts. Also, we can create data or software for performing cyber monitoring, including alerts and detailed forensic analysis, within the games.

These wargames have produced multiple findings regarding cyber security breaches, from the need for leadership to dealing with selfish motives. The participants gained knowledge of cyber incidents based on scenarios from actual breaches. They also demonstrated the necessity of teamwork to meet cyber challenges. Wargaming should continue its current upward trajectory and be recognized as a tool which provides valuable insights to both players and creators.

Douglas A. Samuelson

Theodore T. Allen

Olivia Kay Hernandez

Olivia Kay Hernandez is an entrepreneur and a Ph.D. student at Ohio State University. Her academic interests lie at the intersection of human factors engineering, operations research and applied statistics with cyber security applications. She holds a M.S. in ISE from Ohio State University.

Theodore T. Allen is an associate professor of integrated system engineering at Ohio State University. He is the president-elect of the INFORMS Social Media Analytics Section and a fellow of ASQ. His research is supported by NSF and ARCYBER.

Douglas A. Samuelson is president and chief scientist of InfoLogix, Inc., a small R&D and consulting company in Annandale, Va. A longtime contributing editor of OR/MS Today, he holds a D.Sc. in operations research from George Washington University.

The authors thank the Army TRADOC Analysis Center, ARCYBER and NSF grant 1409214 to Ohio State University for supporting this work. Allen was the principal investigator; Hernandez, a graduate student of Allen’s, was the project lead; Samuelson served as a consultant.


  1. Allen, T. T., Xiong, H., and Afful‐Dadzie, A., 2016, “A directed topic model applied to call center improvement,” Applied Stochastic Models in Business and Industry, Vol. 32, No. 1, pp. 57-73.
  2. Bestard, J.J., 2016, “Air Force Research Laboratory Innovation,” Journal of Cyber Security and Information Systems, Vol. 4, No. 3, pp. 12-17.
  3. Blei, D. M., Ng, A. Y., and Jordan, M. I., 2003, “Latent dirichlet allocation,” Journal of Machine Learning Research, Vol. 3, January, pp. 993-1,022.
  4. Goel, V., 2017 (March 17), “One Billion Yahoo Accounts Still for Sale, Despite Hacking Indictments,” retrieved from: https://www.nytimes.com/2017/03/17/technology/yahoo-hack-data-indictments.html?_r=1
  5. Goel, V., and Perlroth, N., 2016 (Dec. 14), “Yahoo Says 1 Billion User Accounts Were Hacked,” retrieved from: https://www.nytimes.com/2016/12/14/technology/yahoo-hack.html?_r=0
  6. Hämäläinen, J., Sormunen, J., Rantapelkonen, J., and Nikkarila, J. P., 2014, “Wargame as a combined method of qualitative and quantitative studies,” Journal Of Military Studies, Vol. 5, No. 1, pp. 20-37.
  7. Krebs, B., 2016 (Nov. 29), “San Francisco Rail System Hacker Hacked,” retrieved from: https://krebsonsecurity.com/2016/11/san-francisco-rail-system-hacker-hacked/
  8. Lawrence, D., 2017 (March 15), “Here’s How Russian Agents Hacked 500 Million Yahoo Users,” retrieved from: https://www.bloomberg.com/news/articles/2017-03-16/here-s-how-russian-agents-hacked-500-million-yahoo-users
  9. Perla, P.P., 1990, 2012, “The Art of Wargaming: A Guide for Professionals and Hobbyists,” Annapolis, Md.: Naval Institute Press.
  10. Porter, M.F., 2006, “An Algorithm for Suffix Stripping,” Program, Vol. 40, No. 3, pp. 211-218.
  11. Pournelle, P.E., 2017, “Designing Wargames for the Analytic Purpose,” Phalanx, Vol. 50, No. 2, pp. 48-53.
  12. Samuelson, D. A., 2009, “Playing for High Stakes: Wargamers and Cognitive Scientists Seek to Avoid ‘Strategic Surprise,’ ” OR/MS Today, December.
  13. Samuelson, D. A., and Vane, R.R. III, 2015, “Wargamers Explore ‘Forbidden Options,’ ” OR/MS Today, June.
  14. Turnitsa, C., 2016, “Adjudication in Wargaming for Discovery,” Journal of Cyber Security and Information Systems, Vol. 4, No. 3, pp. 28-35.
  15. Weise, E., 2016 (Nov. 28), “Ransomware attack hit San Francisco train system,” retrieved from: https://www.usatoday.com/story/tech/news/2016/11/28/san-francisco-metro-hack-meant-free-rides-saturday/94545998/
  16. Whittaker, Z., 2017 (Feb 15), “Yahoo warning users that hackers forged cookies to access accounts,” retrieved from: http://www.zdnet.com/article/yahoo-warning-users-that-hackers-forged-cookies-to-access-accounts/
  17. Work, R., 2015 (Feb 9), “Memo to Pentagon Leadership on Wargaming,” retrieved from: https://news.usni.org/2015/03/18/document-memo-to-pentagon-leadership-on-wargaming
  18. Zenko, M., 2015, “Red Team: How to succeed by thinking like the enemy,” New York, N.Y.: Basic Books.