2024 ASEE Annual Conference & Exposition

Research-based concept inventories, such as the Force Concept Inventory, the Force and Motion Conceptual Evaluation, or the Energy and Momentum Conceptual Survey have played a pivotal role in designing and evaluating instruction in introductory physics courses. While these concept inventories have helped identify inequity in course instruction and have led to improved pedagogical methods, concerns about their ability to provide actionable feedback for formative assessment remain. In addition, using the same questions for both a pre- and a post- assessment leaves the potential for students learning the correct answers without undergoing conceptual change. To address these issues, we designed a Computerized Adaptive Testing (CAT) platform that utilizes Cognitive Diagnostic Modeling (CDM) for delivering concept inventories through the Learning About STEM Student Outcomes (LASSO) platform. CAT is a modern approach to educational technology that can transform classroom assessment and self-assessment strategies. CAT selects questions using item difficulty and item discrimination to estimate student ability. By selecting questions at an appropriate difficulty for each student, CAT can assess student conceptual understanding with greater accuracy. The addition of CDM allows for an assessment of skill mastery across concept and provides actionable information to instructors to provide individualized instruction even within large-enrollment introductory physics courses. LASSO serves as a centralized platform enabling classes nationwide to access a diverse array of assessment contents and questions aligning with established educational standards, promoting frequent assessment. In this presentation, we describe the design of Mechanics Cognitive Diagnostic (MCD) that uses CAT and CDM and is delivered within the LASSO platform. We also present the results from simulation studies that assessed the ability of the MCD to assess conceptual understanding while maintaining test security and decreasing test length. In addition, we present the results from analyses of student data to identify the underlying skills within conceptual inventories. The amalgamation of CAT with cognitive diagnosis models within the LASSO platform empowers educators to gauge student mastery levels and confidently navigate the subsequent stages of the teaching process.