2023 ASEE Annual Conference & Exposition

Traditionally, students in Science, Technology, Engineering, and Mathematics (STEM) classrooms work on a design project and culminate work with a presentation to their peers. This model may be appropriate to measure student learning, but often does little to directly stimulate student learning because student learning is seen to start at the onset of a project/problem and stop when the assignment is submitted for evaluation. Instead, our study (funded by the NSF Division of Research on Learning) is developing, refining, and testing a protocol in which students evaluate prior work to prime them for learning while designing, through what we call Learning by Evaluating (LbE). This approach includes two important changes to the currently-practiced educational paradigms: 1) actively engaging students—in addition to teachers—in the critique and evaluation process; and 2) students performing the evaluation of work prior to embarking on an assignment—as opposed to the end of an assignment when evaluation traditionally takes place. Anticipated learning outcomes from this experience align with three critical variables in our research design: we believe allowing students to solidify their own understanding of the content, context, and ways of thinking for an assigned project (design mindset) and identifying key features related to work quality that develop evaluation and argumentation skills (critical thinking and reasoning)—both before beginning work on an assignment—will improve students’ performance in future design (student performance).

The first two years of our study use a design-based research approach to guide the design, implementation, and iteration on the protocol for Learning by Evaluating in 9th grade classrooms. By conducting this work in classroom contexts, targeting changes in practice, and collaborating on improvements with practitioners, this setting is well-suited to the design-based research approach. Following this development, we will implement a quasi-experimental design to test the efficacy of the LbE instructional approach.

At the time of submission our study is in its second year and continuing to study the instructional approach through teacher reflection, classroom observation, and initial analysis of students’ critical thinking and reasoning. Our paper and poster will describe three main developments and insights from the project thus far. First, we will describe the initial instructional approach, including connections to theory and epistemic practices in engineering argumentation. Next, we will describe two main trajectories for the LbE comparison experiences and classroom discussion that have emerged from classroom use. Finally, we will reflect on our improvements to the instructional experience based on teacher feedback, students’ use of critical thinking and reasoning, and the logistics of implementing the project. The central focus of this research is to position the student as an evaluator and initiate learning in the design process; the LbE approach has the potential to advance the preparation of critical thinkers in engineering design activities and our developmental approach is ensuring the exploration of ways to optimize this experience.