2026 ASEE Annual Conference & Exposition

A common philosophical discussion among STEM educators is whether students are developing a good conceptual understanding of math, physics, and/or engineering principles, or relying on pattern matching skills to solve long form problems. Building on prior work that looks at student performance on long form exam questions and concept inventory questions in Dynamics, this study applies a standardized rubric to identify points of conceptual misunderstanding in student solutions and how these relate to performance on concept questions.
Three faculty from diverse institutions collaborated to create common exam problems paired with a concept question along with a seven part rubric to evaluate the solutions. The rubric includes the categories of:

1. Coordinate Systems,
2. Free Body Diagrams
3. Acceleration Diagrams
4. Equilibrium Equations
5. Other Equations
6. Algebra
7. Final Answer

For each category, the work is evaluated as correct or incorrect, and incorrect answers are evaluated using the following rubric to identify the error type:
1. Algebraic
2. Units
3. Transcription
4. Calculation
5. Conceptual understanding

This paper presents preliminary analysis of paired data comparing student long form solutions with concept question response. The conceptual questions are scored as correct or incorrect, while the long form solutions are classified using the rubric. The early results indicate that students can correctly answer concept questions while having conceptual misunderstandings in specific parts of their solutions like drawing the Free Body Diagrams and identifying the constraint/kinematic equations. However, over 35% of the students were able to write correct equilibrium equations in their long form solution but answered the corresponding concept question incorrectly.

The use of the rubric provides new insight into the potential points that misconceptions happen in problem solving and how these align, or fail to align, with performance on concept inventories. Both types of skills are considered important for learning topics like Dynamics, and these results further the understanding of how these skills develop and relate for students.