2026 ASEE Annual Conference & Exposition

Engineering students need practice - lots of it. However, the process of creating unique, error-free practice problems in engineering is a time-consuming process. Adaptive problems where the numbers and wording change with each attempt are ideal in providing such practice. Such problems also reduce the potential for copying solutions or generating them from memory.
OpenAI Codex has been used to generate programming problems, solutions, and explanations, and found many outputs to be novel, sensible, and usable without editing. Other efforts include the use of GPT-4 to classify multiple-choice questions by knowledge components (KCs). A knowledge component is a discrete piece of what one must know or be able to do to solve a problem or perform a task in a domain. In educational settings, decomposing tasks into KCs enables instructional systems to track which components a student has mastered and which they still struggle with.
Our research builds upon our prior work leveraging Large Language Models such as GPT-4 to generate adaptive learning modules, including generated questions and detailed solutions. We now introduce knowledge components by developing a classification tool that identifies and maps these knowledge components across lectures and adaptive modules. By understanding which KC are required for more advanced tasks, we can better aid students by building pre-requisite knowledge systematically.
This study will focus on Transport Phenomena, an upper-division mechanical engineering course, which combines knowledge of fluid mechanics, thermodynamics, and heat transfer; thus, many topics are prerequisites. We use a large language model to generate up to 5 knowledge components per task, including problem sets and lecture notes. We have two models: a model that does not have a prior KC list, and the LLM fills knowledge gaps that it deems relevant. In parallel, we construct another model that will operate under constraints, and we will provide it with a predefined KC pool from which it can choose up to five components. Any knowledge component endorsed by our subject matter expert, especially novel ones from the free generation mode, will be incorporated into a repository of validated subskills.
The system will use the KC database to guide retrieval by (a) targeting prerequisite concepts, (b) detecting knowledge gaps in student responses, and (c) recommending adaptive learning modules to bridge those gaps. We will test the effectiveness of the knowledge components generated in this work by comparing the performance of students on targeted questions on the final exam in modules where the KC will be made available to the students and in modules without such help. The midterm exam and quizzes will be used as control. Standard statistical analysis will be used to analyze the data. In addition, we will also study and report the patterns of use among students in order to understand how students interact with AI.