2026 ASEE Annual Conference & Exposition

Engineering design requires making and justifying evidence-based decisions throughout a design process, from early problem framing through validation. Preparing students to engage in these decisions is a central goal of engineering design education; however, how novice designers enact key decision-support activities – analysis, evaluation, and testing – across design phases remains insufficiently understood. This paper presents a qualitative document analysis of capstone design reports from senior-level undergraduate engineering teams to characterize how these activities are enacted across design phases. Analysis is defined as the use of analytical or computational models to examine concept performance; evaluation as the assessment of concepts against metrics, objectives, and specifications; and testing as the examination of how concepts perform in practice [1]. Findings indicate that these activities are infrequently documented during front-end design phases and are instead concentrated in the middle and later stages of the design process. Across reports, analysis and evaluation activities were most often used to support concept comparison and down-selection, frequently through decision matrices such as Pugh charts, while testing was primarily framed around confirming or validating selected concepts rather than informing early design direction. Importantly, there appears to be a missed opportunity to more directly connect analysis and testing activities to the criteria and evidence used within evaluation and decision matrices during concept selection. Taken together, these patterns suggest that novice designers tend to position analysis, evaluation, and testing as downstream selection and verification and validation mechanisms rather than as integrated, iterative activities for early-stage exploration and framing. The findings highlight the need for instructional approaches that more explicitly support the early and continuous use of analysis, evaluation, and testing to strengthen evidence-based design decision-making across all phases of a design process.