2026 ASEE Annual Conference & Exposition

Optimizing Question Bank Size to Influence “Gaming the System” Behavior

Presented at Computers in Education (CoED): Computing Pedagogy & Methods (3 of 8) -- M508C

In a teaching for mastery course, instructors often use question banks to generate multiple random quizzes to let students practice the same concept until mastery. While honest learners effectively master the concepts this way, other students can exploit the no-penalty structure to “game the system” without having to learn. In this paper, we collected five semesters of teaching for mastery classroom data, built a behavior model, and studied how the question bank size influences this “gaming the system” behavior.

We borrowed insights from game theory, such as the Nash equilibrium and Stackelburg game, to model teacher-student interactions. We successfully built the model. To study the relationships, we will vary the model by changing the question bank size and simulate the teacher-student interactions. The Nash equilibrium, computed from the simulation with various learner-gamer ratios, will inform us of the optimal balance between teacher strategies (e.g. the question bank sizes) and student strategies (honest learning or gaming). Using the collected data, we anticipate our simulation will inform us that when there are more gamers than learners in the class, the teacher should favor a larger question bank size to maximize student learning. Using our results, we could identify the optimal question bank size (i.e. how many questions to have in the bank if all of the students are gamers). Due to the limited amount of data collected, we chose linear regression as our behavior model. In the future, we aim to collect more data and build a more sophisticated model.

In summary, we address a common challenge in teaching for mastery: how to mitigate the “gaming the system” behavior while still encouraging learning. We conducted an exploratory study using five semesters of data to build a teacher-student interaction model and employed game theory to identify the optimal question bank size. Knowing the gamer and learner distribution of an institution, our approach provides insight for educators on how to handle teaching for mastery exploits.

Authors
  1. Nicole Qian Bucknell University [biography]
  2. Lea Wittie Bucknell University [biography]
Download paper (961 KB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.