INNOVATIVE EDUCATION: Gaming - Learning at play

Gaming: Learning at play

An inside look at the design and analysis of games – serious teaching tools for poorly structured problems.

By Elizabeth Bartels

The government, businesses and academia are increasingly interested in teaching about poorly structured problems. Gaming’s ability to capture key dynamics in immersive and memorable ways has made it an increasingly popular tool to tackle such teaching. The key to realizing educational gaming’s potential is through thoughtful design of the game and careful analysis of its effectiveness after it is played. 

First we lay out how education games differ from games built for other purposes that have been more widely and deeply documented. We then briefly explore the use of gaming for education in several fields, and why it is seen to be an effective teaching tool. Finally, we present some basic rules of the road to help educational gamers to think systematically about their design choices.

Differing Purposes of Games

As depicted in Table 1, the purpose of a game can be to teach existing knowledge or to produce new knowledge. Games can also be defined by whether the subject of the game is a structured or unstructured problem. The intersection of these factors delineates four basic purposes of gaming: discovery, analysis, training and education. 

Discovery games seek to generate new hypotheses and variables of interest about an unstructured problem. Games of this type can include investigations of new problems, consideration of potential future conditions and rare events such as black swans [1]. In contrast, analytical games seek to help us understand phenomena that are relatively well structured. Thus, these games often seek to illustrate how human decisions interact with well-defined (often physical) phenomena, like weapons system performance. The third grouping involves games designed to train individuals about specific, well-understood tasks. These games aim to provide a simulated environment for individuals to rehearse particular tasks, and are often treated as controlled “practice” sessions for individuals to build up skills. 

Finally, educational games seek to instruct participants about poorly structured phenomena. These games focus on conveying an understanding of issues in which there is not a consensus regarding relevant norms and values, and for which there is a great deal of uncertainty. Thus, this category of games involves the inherently tricky proposition of teaching topics in which the instructors do not have uncontested answers.

Why Gaming Works for Education

Educational gaming’s potential

The use of gaming to support education is not new, but it has received more attention and more diverse support in recent years. Within the field of education, gaming has been referenced as a method to support the related principles of active learning [2], experiential learning [3] and student-centric learning [4]. In parallel, research on early childhood development has also focused on the role of play in learning [5]. The business community has also identified gaming as a key means of educating students and leadership about strategic problems and decision-making [6] and encouraging innovation within the workforce [7]. Finally, military education has long seen games as a valuable teaching tool, and in recent years gaming has been highlighted as a way to make education more realistic and operationally relevant [8]. 

However, the difficulties inherent in measuring learning about unstructured problems have meant that there is a limited amount of systematic, experimental research that assesses gaming’s specific benefits compared to other pedagogical tools. Furthermore, because games are time and resource intensive, many active, experiential and student-centric learning studies have focused on other techniques. Similarly, the majority of work describes particular games, rather than laying out how games can be designed to be more effective teaching tools. For example, while there is a rich canon of work in the field of international relations that describes specific games that instructors have found valuable in their teaching practices, there is little general guidance about how to construct useful games. 

We need a more targeted understanding of when and how games are effective educational tools, and more thinking about what these findings can contribute to best practices and standards in game design. Meanwhile, anecdotal evidence suggests two primary hypotheses of how games teach:

1. Games are models that players can relate to directly. They are helpful not only because they provide simple and memorable explanations of phenomena, but also because they manifest that explanation through compelling stories and experiences students can interact with directly. This makes games particularly effective when working with students who are used to learning on the job, as it can make abstract material relevant, memorable, and applicable in a way that abstract models presented directly often are not.

2. Games are immersive, allowing limited replication of emotional and psychological experiences. This can make them effective ways to practice decision-making because they reproduce some of the emotional stakes of real life, without imposing many of the risks that decisions carry in the real world.

Game Design for Education

Regardless of purpose, we can think of games as having five key elements. First, objectives define what is to be learned by playing the game. Second, the environment provides artificial context for the game’s decision. Third, players are assigned to represent the roles of key decision-makers and stakeholders in the phenomenon of interest. Fourth, rules define how players can make decisions, and how those decisions will affect the environment. Finally, analysis draws lessons learned from the game to convert the experiences from the time-bound event into durable knowledge.

Goal of Game

An alternative way to understand these elements is that educational games help students understand a model of the phenomena of interest by instantiating the model’s structure and mechanism through the environment, rules and player roles. Thus, the players are assigned to represent the key actors defined by the model, the environment illustrates the key contextual factors the model implies shape decision-making, and the rules define the key tools and limitations of decision-making as depicted by the model. Finally, analysis determines how well the game was able to illustrate the model. 

By this understanding, games are successful to the extent that they are able to recreate key structures, actors and interactions outlined in the model in the design elements. Incorrect or unintentionally omitted key elements will distort the model and often lead students to draw the wrong conclusions from game play. Conversely, inclusion of superfluous elements will add complexity that can confuse students and overwhelm the limited time available in the classroom.

Educational games exhibit particular requirements and limitations that impact how these elements manifest themselves in games. In the paragraphs that follow, we describe some of the major design considerations for each of these five elements, discuss some of the most common design choices, and briefly describe some of the design pitfalls that can limit games’ effectiveness.

Objectives. The first element of any educational game is clearly stated objectives that delineate what students are intended to learn through their participation in the game. These are important to define so that subsequent choices about game design are done in a thoughtful way, and so that there are clear criteria available to determine if a game was successful. Failure to define objectives clearly can lead to games that are poorly designed or that do not teach the desired information. Game objectives should also always be integrated into the broader course of learning. Information presented as part of the game should either complement other information on the topic presented in the course or be presented in a way that explicitly acknowledges tensions between competing theories. Failure to connect the learning objectives of the game to the course of study as a whole can produce games that feel irrelevant to participants.

Variation in game objectives is generally determined by the type of problem under consideration and the level of mastery students are expected to demonstrate. The more unstructured the problem is, the more the objective of the game tends to focus on exploration of the problem. Somewhat more structured problems tend to focus on gaining a better understanding of the competing interests that make poorly structured problems difficult to resolve. At the same time, as student mastery increases, objectives might shift from application of concepts to analysis, synthesis and evaluation of ideas.

Finally, while games should always seek to teach about focused and concrete phenomena, the unstructured nature of the problems tackled by educational games means that there is usually space for more than one correct answer. Thus, educational games are often more about the process of decision-making rather than the specific decision that is ultimately made by players. Attempts by the instructor to force students down a track toward a preordained “correct” outcome will often lead students to feel tricked. This can cause students to reject the information conveyed by the game (thus preventing learning), the game’s process (prompting dysfunctional behavior), or both.

Environment. The second key element of educational games is the setting and context in which the game’s decisions are made. This includes both the selection of a particular setting (for example, does the game depict insurgency in Peru, Iraq or a made-up country) and the particular elements of that setting that are included or excluded from game materials.

The selected environment must be appropriate for demonstrating the lessons the instructor seeks to convey. Just as the selection of traditional texts on a syllabus should illustrate the problem under study, so too should the environment of the game. Choosing an appropriate environment is also tightly linked to the level of mastery students are expected to demonstrate. For example, exploration of a new concept is generally best illustrated with a clear, minimally ambiguous example. It can also be helpful when exploring a new concept to use a familiar case to reduce the number of factors the student must learn simultaneously. In contrast, more advanced students can often benefit from gaming to explore novel cases that allow them to add complexity and richness to their understanding of a topic. Here, the literature on case studies [9] can be a particularly helpful guide to considering what setting might be most valuable. 

The type and amount of information about the setting that is available to players is also critical. As a general rule of thumb, the more parsimonious the model being taught the more abstract and simple it is possible to be in depicting the environment. This is because the more guidance the model provides about what is and is not important in contributing to the phenomenon of interest, the easier it is to narrow down the information to be presented to only the most relevant facts. That said, the unstructured problems of educational gaming very rarely have such uncontested, parsimonious models as their focus. Games also often explore decision-making environments that are characterized by information defect or overload. In this case, student frustration is a key element of the game, and should be explicitly discussed as a key point of learning.

Players and roles. The third key element is the way the game represents key actors and their attributes. In considering this element of a game, we are interested both in how actors are defined and presented in game materials and the players who are assigned to take on these roles during game play. Unlike games for research, often educators have very little ability to affect the backgrounds and skills of their students, and thus cannot control the selection of participants in the game. This limitation makes it particularly important that roles are written in a way that accommodates the students in the room. 

There are three common variations on role design in educational games. The first seeks to help players better understand how the role they occupy fits into a broader system or institution. This type of game is often seen in senior leader education, when individuals take on new responsibilities and must understand how their duties and equities relate to broader decision-making. A second common variant places students in a role they hope to occupy in the future, to highlight the skills they will need to develop moving forward. For example, military education frequently uses games to help officers think through the actions and responsibilities of those higher up in the chain of command in order to identify skills to improve on and to understand better the concerns of senior decision makers. 

A third option assigns players to roles they will never occupy in order to allow students to better understand the motivations, tools and limitations of other actors. The best-known variation of this is “red teaming,” in which players take on the perspective of an enemy or competitor to improve understanding and anticipating the real actors’ decisions.

Rules. The fourth element of games is the set of rules that state what players can and cannot do, and what if any effect play decisions will have on the environment and other players. The most common approach to rule sets design is to try to mimic the processes and constraints of the system under study. Games can also be designed to present students a formalized set of rules that they can use to compare and refine their own mental models.

Education game rules can vary in complexity, form, specificity and medium. In much the same way as the complexity of the environment, complexity of the rules is generally dictated by the phenomena of interest and the parsimony of the model the game instantiates. Game rules can range from very formal and rigid (such as is common in board games and computer games) to very flexible (such as in seminar games). Specificity may range from a general description of their role and goals to a step-by-step process that students are tasked to follow. Finally, rules can be communicated to students through a wide range of media, although written instructions are most typical. 

Regardless of this variation, rules work best when they are carefully calibrated to the depiction of the roles and environment. This ensures that all components of the game feel integrated to players and can serve as a useful check for designers to spot inconsistencies with either the model or earlier design choices.

Analysis and assessment. Educational games are generally analyzed from two different perspectives: what students learned and how useful the game was in teaching. Generally, the first of these perspectives means determining whether students gained a better understanding of the topics delineated by the game’s objectives by participating in the event. For the second, the key question is whether another tool (whether a differently designed game or another pedagogical approach) would have been a better teaching tool. “Better” can include issues ranging from efficient use of classroom time, time spent by both instructor and students in preparation, and the content of what was learned. 

In both cases, measuring what learning has occurred has proven to be a substantial challenge. Traditional educational research methods like pre- and post-testing of the material often are not well suited to capturing learning about complex, poorly structured issues. Self-assessment by students is a common method, but it can sometimes reveal only that the students read the objectives and therefore knew what they were intended to learn, rather than what they actually learned. Compounding the issue, many educational gamers state that learning from games often happens substantially after the game occurs, when an experience later in life connects with the experience of the game, allowing for the synthesis of key ideas. Capturing this experience requires long-term data collection that is generally not feasible. 

While these considerations are not trivial, the most common failure of analysis in educational games is simply that gamers don’t perform it. This can occur when game objectives are not clearly defined or when games are not taken seriously, leading the designer to cut corners and create games that lack needed rigor. Documenting “hotwash” (immediate post-game review) discussion, student self-assessment of learning, faculty observations, and pre- and post-testing can all provide important evidence to assess student learning and the game’s performance. Considering the unstructured nature of the problems addressed in educational gaming, it can also be advantageous to collect information on a broad range of issues, as often student learning extends beyond the key topics outlined by the objectives. Techniques such as open survey questions and semi-structured interviews can allow for this information to be elicited and captured.

Conclusions

Educational gaming offers great potential for preparing leaders to make better decisions. However, games are only effective when they are thoughtfully designed and the results are well analyzed. As educational gaming gains popularity, ensuring that best practices on both design and analysis are compiled will be critical to improving the quality of practice over time. As a starting point, it is important to recognize how educational games are similar to and different from gaming for other purposes, acknowledge what we do and do not know about gaming as a form of teaching, and document how the practice of game design should be practiced to ensure the best possible educational results.

Elizabeth Bartels (ebartels@caerusassociates.com) is a senior associate at Caerus Associates. An analyst and educator, Bartels specializes in understanding groups engaged in political contestation and violence. Prior to joining Caerus, Bartels led teams to design educational and analytical strategic wargames at the National Defense University, focusing on games to improve participants’ understanding of strategy in irregular warfare environments and their effects on populations. 

The author thanks Doug Samuelson for his encouragement and thoughtful editorial suggestions.

References

1. Peter Perla, “Wargaming and Analysis,” (presented at the Military Operations Research Society’s workshop on Wargaming and Analysis, Chantilly, Va., Oct. 16-18, 2007) available online (http://www.mors.org/UserFiles/file/meetings/07wa/perla.pdf) and Peter Perla, “The Art of Wargaming 2nd Edition,” History of Wargaming Project, 2012.

2. Jeffrey S. Lantis, Lynn M. Kuzma, and John Boehrer, ed., “The New International Studies Classroom: Active Teaching, Active Learning,” Lynne Rienner Publishers: Boulder, 2000.

3. John P. Hertel and Barbara J. Mills, “Using Simulations to Promote Learning in Higher Education: An Introduction,” Stylus Publishing: Sterling, Va., 2002.

4. Jeffrey Froyd and Nancy Simpson, “Student-Centered Learning Addressing Faculty Questions about Student-Centered Learning,” accessed July 18, 2014 (http://www.jfn.ac.lk/OBESCL/MOHE/SCL-articles/Academic-articles/16.SCL-Froyd.pdf).

5. National Association of the Education of Young Children, “Play,” accessed July 16, 2014 (http://naeyc.org/play).

6. Doug Samuelson, “Wargaming for Fun and Profit: A New Tool for Competitive Analytics,” Analytics Magazine, March/April 2001, and Benjamin Gilad, “Business War Games,” The Career Press, 2009.

7. Michael Schrage, “Serious Play: How the World’s Best Companies Simulate to Innovate,” Harvard Business Review Press: Cambridge, Mass., 1999.

8. TRADOC Pamphlet 525-8-2, The United States Army Learning Concept for 2015, Version 0.71 (Washington, D.C.: U.S. Government Printing Office [GPO], November 2010), pg. 8, 20, 22, 25, 27 and 29.

9. I have found Alexander George and Andrew Bennet, “Case Studies and Theory Development in the Social Sciences,” MIT Press: Cambridge, Mass., 2005, to be particularly helpful.