2023 ASEE Annual Conference & Exposition

Oral Assessments as an Early Intervention Strategy

Presented at Assessment-Driven Practices in ECE

Oral assessments, i.e., one-on-one interview-style questioning by an instructor, have been shown to be powerful pedagogical tools. Their main benefits include the ability to assess conceptual mastery in depth due to their adaptive dialogic nature, in addition to improving students’ verbal skills and serving as a tool to support academic integrity. However, assessments not only play an important role in measuring the level of students' understanding, but the assessment method also guides students' learning strategies. As such, oral assessments can serve as an important driver for students to pursue conceptual knowledge. While these dialogic assessments may exhibit challenges regarding potential bias, reliability and validity, past research has shown that with careful training and crafting these can be largely overcome. However, the main impediment towards more widescale adoption is the issue of scaling with larger class sizes, due to its reliance on one-on-one interactions between students and members of the instructional team.

To overcome the scaling issue, we have implemented and investigated an approach in which the oral assessment is only offered to a subset of students, specifically those who failed an early written exam in the course. This approach is rooted in the work on early intervention strategies. The idea is to focus on at-risk students. In this context, we do not consider the oral assessment primarily as being part of a summative assessment strategy. Instead, it is designed to be a touch point for a meaningful one-on-one interaction between a student and a member of the instructional team. The value of early interventions for at-risk students is to increase connectedness to instructional staff and resources, and student engagement and self-efficacy. The oral assessments were implemented explicitly with this focus. We also considered additional benefits, such as serving as formative assessments for the students to reflect on their level of conceptual mastery and learning strategy.

Our study was implemented in an intro to electrical engineering course, which serves as the gateway to the core curriculum. For the purposes of this intervention, the first written exam, administered at the start of week 4 of the term, was used as the tool to identify at-risk students. In this paper, we will discuss the effectiveness of this intervention, based on qualitative and quantitative data. Student feedback, obtained via a set of three surveys, shows that students find value in the oral assessment and that it increases their self-efficacy and their connectedness to the instructional team, lowering barriers to seek help. While the intervention group is too small to reach statistical significance, results hint at performance gains as well, especially when students also approach the oral assessment as a learning opportunity. These results are encouraging, as they suggest that the intervention can capture the benefits of oral assessments, while being more scalable and more targeted towards at-risk groups.

Authors

Curt Schurgers University of California, San Diego [biography]

Curt Schurgers is an Teaching Professor in the UCSD Electrical and Computer Engineering Department. His research and teaching are focused on course redesign, active learning, and project-based learning.
Dr. Saharnaz Baghdadchi University of California, San Diego [biography]

Saharnaz Baghdadchi is an Associate Teaching Professor at UC San Diego. She is interested in scholarly teaching and employs active learning techniques to empower students to attain an expert level of critical thinking. Her expertise facilitates students' journey towards connecting facts with practical knowledge to tackle intricate engineering challenges. She excels in crafting innovative assessments and explores their impact on enhancing students' learning outcomes and fostering an inclusive educational environment.
Dr. Alex Phan http://orcid.org/0000-0003-2489-2886 University of California, San Diego [biography]

Dr. Phan received his Ph.D. in Mechanical Engineering from the University of California San Diego with a specialization in medical devices. He is currently an instructor for the Department of Electrical and Computer Engineering focusing on hands-on education.
Dr. Huihui Qi University of California, San Diego [biography]

Dr.Huihui Qi is an Associate Teaching Professor in the department of Mechanical and Aerospace Engineering, at the University of California San Diego.
Dr. Maziar Ghazinejad University of California, San Diego [biography]

Maziar Ghazinejad is an assistant teaching professor in Mechanical and Aerospace Engineering Department at UC San Diego. He received his Ph.D. in mechanical engineering from UC Riverside in 2012 and holds M.S. degrees in mechanical and electrical engineering. Prior to his appointment at UCSD, he served as an assistant professor and graduate coordinator at California State University Fresno, where he received the provost faculty award in 2018. His teaching repertoire includes engineering mechanics, materials science, advanced manufacturing, and engineering design. He has also developed new classes on microanalysis, design, and nanoengineering.
Ghazinejad's research on manufacturing and application of nanomaterials has produced several key journal publications, including featured cover articles, and received an Energy & Efficiency Developments (DEED) award from the American Public Power Association. He is a member of the American Society of Mechanical Engineers (ASME), Materials Research Society (MRS), American Society of Engineering Education (ASEE), and the International Society for Optics and Photonics (SPIE), where he serves as a conference chair and editor.
Dr. Minju Kim http://orcid.org/0000-0001-5878-7350 University of California, San Diego [biography]

Minju Kim is a postdoctoral scholar at the Engaged Teaching Hub at the UCSD Teaching+Learning Commons. Minju received her Ph.D in Experimental Psychology at UC San Diego. With Engaged Teaching Hub, Minju has designed TA training materials for oral exams and have conducted quantitative analysis on the value of oral exams as early diagnostic tool (Kim et al., ASEE 2022). Minju is interested in designing assessments that can capture and motivate students' deep conceptual learning, such as oral exams and the usage of visual representations (e.g., diagrams and manual gestures).
Marko V. Lubarda http://orcid.org/0000-0002-3755-271X University of California, San Diego [biography]

Marko V. Lubarda is an Assistant Teaching Professor in the Department of Mechanical and Aerospace Engineering at the University of California, San Diego. He teaches mechanics, materials science, design, computational analysis, and engineering mathematics courses, and has co-authored the undergraduate textbook Intermediate Solid Mechanics (Cambridge University Press, 2020). He is dedicated to engineering pedagogy and enriching students' learning experiences through teaching innovations, curriculum design, and support of undergraduate student research.

Download paper (1.15 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session