Using simulation games to assess learning
The increased use of simulation games in higher education and the advent of ChatGPT present a major opportunity to rethink student assessment. As universities worry about how to implement authentic assessment in the era of AI, simulation games provide novel ways for students to demonstrate their understanding of key course topics and their application in the real world. This article gives practical guidance on opportunities to assess learning with simulation games in the new age of education.
Game scores = course grades?
The obvious way to use simulation games for assessment of learning is to use students’ game scores as inputs into grading. However, this is not as simple or effective as it may seem at first sight. First, the instructor needs to determine the winning criteria for the game. This is clear in some simulation games, but not all. For example, in a business simulation, do students need to maximize final year profit, cumulative profit, the share price, or something else? Making the winning criteria clear, and not changing it during the simulation game exercise, is a first step in reducing student anxiety and getting them engaged.
Secondly, students want to know how a simulation game is incorporated in grades they receive for a course. Students can experience excessive fear of failure and feelings of inequity if their course grade may be negatively impacted by performance in a simulation game exercise. Students may start criticizing the game, especially if the model underlying the simulation does not produce the results they expect. Instructors will be expected to explain students’ results in a level of detail that they may not be comfortable with. In the end, evaluating the model behind a simulation can be a useful learning experience, but arguing about it for the sake of getting a higher grade is not productive.
Team dynamics can also suffer, as some students may display (or be accused of) ‘free riding’ behavior. All this leads to a setting with ‘winners’ and ‘losers’ from the game experience, which is not conducive to learning.
Advocates of including game scores as an element in course grading argue that it increases engagement, motivation and accountability. Although this may be true, these benefits need to be set against the concerns. Usually, symbolic rewards, like a trophy or a small gift for strong performers, can generate as much excitement as grades. In the end, if the decision is made to include the game score as a component of a course grade then the recommendation is to make the weight of the game score relatively small, so students have opportunities to demonstrate their learning in others ways as well.
Learning reflections
Learning reflections offer better opportunities to assess learning, since they focus on the learning that has taken place rather than just the game scores. Reflections can be structured as individual or group projects, in either written or presentation format. Each of the four resulting settings (individual-written, individual-presentation, team-written, team-presentation) can be effective. In practice, individual student presentations are rarely used since they can take up excessive class time, although for large classes they could be recorded and evaluated by the instructor outside class.
If a learning reflection is used for assessment, students need clear guidelines and a rubric. A generic set of questions can include:
- What was your strategy in the game and how successful was it? Which decisions worked out well or badly for you? Why?
- What did you learn by playing from the simulation game exercise? How does this learning relate to the topics covered in the course?
Students may also be asked to evaluate the simulation game and the model underlying it, providing a further opportunity to develop critical thinking skills.
These questions give students the chance to show that they understand the simulation, have learned from it and can link this learning to both the course topics and to the real world. Addressing these topics is a personal and context specific exercise in which ChatGPT is not very useful.
Exams and quizzes
Exams and quizzes can be used with simulation games for both formative and summative assessment purposes. Within Sim Institute’s games there are now three multiple choice quizzes that instructors can make available to their students:
- A formative test, in which questions are selected based on the decisions and results of each individual student in the simulation game. Each time, the system selects questions that reinforce knowledge and skills in areas that need attention. For example, in the Sustainability Management Simulation: Net Zero, if emissions from the hotel’s purchasing activities are not reduced during the game, quiz questions can test and build knowledge about supply chain emissions. Students can play the same simulation several times and experience a different set of questions after each iteration of the game.
- A summative test with a standard set of questions that summarize learning from the game.
- A summative test with questions selected by the instructor from a test bank that is available in the simulation’s facilitator interface. Questions are organized into knowledge areas, so it is easy to distribute questions among them.
These tests are all designed to link the simulation to the simulation’s learning objectives and to key course topics in relevant disciplines.
If a simulation is an important part of a course, then reflections and quizzes can both be used for assessment of learning (and there may even be a small element determined by game scores for instructors who insist on this practice!). In the end the selection of assessment methods will depend on individual circumstances and preferences.
With universities and schools rethinking how to assess student learning, simulation games offer a great deal of untapped potential to help students learn and to measure student progress along their learning journey.
