2025 ASEE Annual Conference & Exposition

This study uses fixed and variable video game types to measure pretest sensitization as a proxy for repeated and varied threat test scenarios in system performance testing of air and missile defense systems. The pretest sensitization phenomenon exists when repeated exposure to a test condition influences the participant's response. Research shows that air and missile defense development correlate with video games, resulting in similar interfaces and computer operating environments. This study demonstrates the need to vary the scenarios to provide accurate system performance results supporting programmatic decisions. The study uses a three-factor nested factorial design of experiments. Independent variables are: video game type (IV1), title nested within game type (NV1), and completion: first or replays (IV2). Time to complete the game is dependent variable. Five video game titles are sampled for each type, comparing average first completion(s) with replay(s) to detect a decrease in the time to complete the game with repeated exposure, indicating pretest sensitization. The data downloaded from www.HowLongToBeat.com represents a snapshot of results when downloaded. Using all completionist data with notes results in a sample size of 1,598. The findings confirm pretest sensitization with repeated exposure, impacting the accuracy of system performance and adding risk to programmatic decisions. Hence, the test scenarios in Department of Defense (DoD) or engineering management training must be varied to help determine system performance accurately. This is important because system performance results dictate future course of action in engineering management or DoD decision-making. Such results inform acquisition decisions such as further funding and development, program canceling, and fielding decisions.