2023 ASEE Annual Conference & Exposition

Purpose: The University of Georgia Department and Leadership Teams for Action, or DeLTA, is an NSF project that aims to help students develop STEM knowledge and skills. The project brought together more than 50 University of Georgia faculty members in science, technology, engineering, and math to collaborate on a comprehensive research project that seeks to transform STEM education on campus and at research universities nationwide. To facilitates this process, seven faculty level Instructional Action Teams (IATs) were created, and Engineering was one of such teams. This paper presents the key activities and findings conducted under the two key projects by the engineering IAT. The purpose of the first project was to investigate how Test Blueprints enhance assessment for different engineering courses. The second project evaluated student self-assessment as a tool to enhance students learning experience.
Methods: The Test Blueprint project involved Five engineering faculty who teach 5 independent courses. Various Test Blueprint models were reviewed, and each faculty developed a suitable Test Blueprint to evaluate assessment instruments they had used for their course prior to participating in the project. The Test Blueprints were strictly based on the learning outcomes of each participating course. The ABET criteria on student learning outcomes played a significant role in developing the outlines of the Test Blueprints for this study. The participating faculty compared their Test Blueprints to their previous exams and identify how adequately their prior course assessment have included the major student learning outcomes. The Blueprints were then used to develop future assessments.
Nine faculty members were involved in the self-assessment study. After reviewing several self-assessment models, faculty decided to adopt models that suited the mode of their class. Assessment and analysis methods varied from faculty to faculty. However, the group developed a unified self-assessment questionnaire that was administered by all the participating faculty at the end of the study period.
Results. The study on Test Blueprints provided a guided way of properly preparing assessment material to capture the intended learning outcomes of the course. Test Blueprints clearly showed across all courses that, faculty were missing some important learning outcomes when their assessments were not guided by a Test Blueprint. It was also evident that Test Blueprints varied significantly depending on the nature and the delivery mode of the course. On self-assessment, across the seven courses, the results of student perception on self-assessment were consistent. The items with the highest ranked means include doing self-assessment to improve on future assignments (5.52/7), using self-assessment to avoid mistakes made in previous assignments (5.91/7) and understanding the connection between course materials (5.97/7). The least ranked item is the willingness to do self-assessment without any incentive (3.81/7). The results show that self-assessment kept the students engaged in the course, but they will only self-assess if incentivized to do so. Implications of both interventions for enhancing learning and pedagogy are discussed.