2024 ASEE Annual Conference & Exposition

Assessment plays a critical role in students' learning, as it provides feedback to both instructors and students about the learning progress, and potential adjustment needed for both teaching and learning strategies. Often, students’ learning strategies are steered by the exams. Some students may have a good understanding of the basic concepts, but not be able to apply them in a complex application situation; some students may have a good conceptual understanding, but lack of rigor on procedural knowledge. Being able to identify such areas of improvement is important for students to adjust their next-step learning approaches.
The strength and weakness identification, and adjustment of learning strategies requires effective engagement in metacognition. But it is not rare that students often fail to practice metacognition and reflect on their learning strategies to make the appropriate adjustment as needed. At times, students do not clearly understand whether their unsatisfactory exam results are due to time constraints of the exam, or their knowledge or problem-solving skills need to be improved. Also, some students have a difficult time understanding if they lack understanding of their procedural knowledge or individual concepts, or they lack conditional knowledge-when to use isolated concepts together, or problem-solving skills. Thus, assessment methods that can promote students’ metacognition will be important.
Both in-class (closed-book) exam (ICE) and take-home (open-book) exams (THE) have been widely used for engineering classes. Each format provides its advantages and disadvantages. ICEs appear to be a more effective tool for testing the lower levels of Bloom’s taxonomy scale such as remembering , and are associated with less student cheating . On the other hand, THEs appear to be more productive for developing and testing higher levels of the Bloom’s taxonomy scale (e.g. analyzing and evaluating), as the student can invest their energy into deeper engagement and understanding rather than memorizing the material.

The impact of the combination of in-class and take-home exams on helping engineering students’ engagement with metacognition has rarely been studied. In this paper, the impact of the use of a combination of in-class and take-home exams, along with post-exam wrappers on students' metacognition, has been studied. The assessment structure has been implemented into 2 engineering mechanics classes (Statics and Mechanics of Materials) in an public research university in the United States. Between the two classes, a total of 220 students were included in this study. The exams were mindfully designed to serve distinct purposes: the closed book in-class exams were timed, and focused on the isolated concepts without in-depth problem-solving, with no access to books or notes unless necessary equations were given; the open-book take-home exams, on the other hands, were un-timed and aimed to test students higher-order thinking, problem-solving skills using many concepts together.
Students’ exam performance (3 midterm quizzes and 1 final exam), exam wrappers' open-ended responses, and end-of-quarter survey results are studied collectively. From the exam wrappers' open-ended responses, and end-of-quarter survey questions, students’ perceptions about how the closed-book in-class and open-book take-home exams help them to identify the strength and weaknesses of their learning progress and their learning strategies differently. Regression analysis is used to understand the relationship between exam performance progress and their perception of the exams’ impact on their metacognition and learning strategies. A comparison of final exams performance (difficulty level calibrated) between the previous years' cohorts and the cohorts with the two-part exams is made, showing the students with the two-part exams performed better than the previous cohorts with only closed-book in-class exam or open-book online timed exams.