2023 ASEE Annual Conference & Exposition

Board 400: The impact of Oral Exams on Engineering Students’ Learning

Presented at NSF Grantees Poster Session

This project aims to enhance students’ learning in engineering courses through oral exams. The adaptive dialogic nature of oral exams provides instructors an opportunity to better understand students’ thought process, thus holding promise for improving both assessment of conceptual mastery and students’ learning attitude and strategies. However, the issues of oral exam reliability, validity, and scalability have often been cited as important challenges.. As with any assessment format, careful design is needed to increasethe benefits to student learning whilereducingthe potential concerns. Compared to traditional written exams, oral exams have a unique design space, which involves a large range of parameters, including the type of oral assessment questions, grading criteria, how oral exams are administered, how questions are communicated and presented to the students, how feedback were provided, and other logistical perspectives such as weight of oral exam in overall course grade, frequency of oral assessment, etc.

The purpose of our project is to create a framework to integrate oral exams in core undergraduate engineering courses, complementing existing assessment strategies. Due to our focus on these core undergraduate courses, addressing the issue of scalability to large class sizes has been one of the primary focal points. Our solution revolves around the involvement of the entire instructional team (instructors and instructional assistants) in the assessment process. However, addressing scalability, brings the need to address bias and reliability back into focus. As such, our project is built upon two pilars: (1) create guidelines for instructors to build oral assessment implementations within a systematic design space exploration to maximize student learning outcomes; and (2) create a training program to prepare both faculty and teaching assistants to effectively and fairly administer oral exams. The project implements an iterative design strategy using an evidence-based approach of evaluation. The effectiveness of the oral exams isevaluated by tracking students’ performances on conceptual questions across consecutive oral exams in a single course, as well as across other courses.

In last year’s poster, we shared our progress in exploring the oral exam design space, developing Instructinoal Assistants training materials and project assessment instruments.In this year’s poster, we will present our findings from the first two years of the study: (1) An oral exam design guideline generated from a reflective and explorative dataset from 10 different engineering class oral exams implementation; (2) TA training methods; (3) students’ perception on oral exams; (4) instructors and teaching assistants experience and perceptions; (5) how to prepare students for orla exams.

Authors

Dr. Huihui Qi University of California, San Diego [biography]

Dr.Huihui Qi is an Assistant Teaching Professor in the Department of Mechanical and Aerospace Engineering at the University of California, San Diego.
Dr. Minju Kim http://orcid.org/0000-0001-5878-7350 University of California, San Diego [biography]

Minju Kim is a postdoctoral scholar at the Engaged Teaching Hub at the UCSD Teaching+Learning Commons. Minju received her Ph.D in Experimental Psychology at UC San Diego. With Engaged Teaching Hub, Minju has designed TA training materials for oral exams and have conducted quantitative analysis on the value of oral exams as early diagnostic tool (Kim et al., ASEE 2022). Minju is interested in designing assessments that can capture and motivate students' deep conceptual learning, such as oral exams and the usage of visual representations (e.g., diagrams and manual gestures).
Dr. Carolyn L Sandoval University of California, San Diego [biography]

Dr. Sandoval is the Senior Director of Instruction and Pedagogy for the Teaching + Learning Commons at the University of California, San Diego. She earned a PhD in Adult Education-Human Resource Development. Her research interests include adult learning and development, faculty development, qualitative methods, educational equity.
Zongnan Wang University of California, San Diego [biography]

Zongnan is currently a 2nd-year undergraduate student with a major in mechanical engineering and minor in Data Science. He has been assisting Dr. Huihui Qi at the Engineering Pedagogy and Design Lab, with the work of processing and analyzing data from survey responses to study the impact of oral exams on students' learning.
Prof. Curt Schurgers University of California, San Diego [biography]

Curt Schurgers is a Teaching Professor in the UCSD Electrical and Computer Engineering Department. His research and teaching are focused on course redesign, active learning, and project-based learning. He also co-directs a hands-on undergraduate research program called Engineers for Exploration, in which students apply their engineering knowledge to problems in exploration and conservation.
Marko V. Lubarda http://orcid.org/0000-0002-3755-271X University of California, San Diego [biography]

Marko V. Lubarda is an Assistant Teaching Professor in the Department of Mechanical and Aerospace Engineering at the University of California, San Diego. He teaches mechanics, materials science, design, computational analysis, and engineering mathematics courses, and has co-authored the undergraduate textbook Intermediate Solid Mechanics (Cambridge University Press, 2020). He is dedicated to engineering pedagogy and enriching students' learning experiences through teaching innovations, curriculum design, and support of undergraduate student research.
Dr. Saharnaz Baghdadchi University of California, San Diego [biography]

Saharnaz Baghdadchi is an Associate Teaching Professor at UC San Diego. She is interested in scholarly teaching and employs active learning techniques to empower students to attain an expert level of critical thinking. Her expertise facilitates students' journey towards connecting facts with practical knowledge to tackle intricate engineering challenges. She excels in crafting innovative assessments and explores their impact on enhancing students' learning outcomes and fostering an inclusive educational environment.
Xuan emily Gedney
Dr. Alex M. Phan http://orcid.org/0000-0003-2489-2886 University of California, San Diego [biography]

Dr. Alex Phan is the inaugural Executive Director for Student Success in the Jacobs School of Engineering at UC San Diego. Prior to his appointment, he has served as an engineering instructor teaching across multiple divisions, including the Jacobs School of Engineering (Dept. of Electrical and Computer Engineering, Dept. of Mechanical and Aerospace Eng., Dean's Office Unit) and UC San Diego Division of Extended Studies. His teaching interests and expertise are in experiential learning, holistic education models, active learning environments, and metacognition. In his current role, he leads the IDEA Student Center, a prolific student-centered resource hub at the Jacobs School that serves as a model for student success units across the country.
Dr. Nathan Delson eGrove Education [biography]

Nathan Delson, Ph.D. is a Senior Teaching Professor at the University of California at San Diego. He received a PhD in Mechanical Engineering from MIT and his interests include robotics, biomedical devices, product design, engineering education, and maker spaces. In 1999 he co-founded Coactive Drive Corporation (currently General Vibration), a company that provides force feedback solutions. In 2016 Nate co-founded eGrove Education an educational software company focused on teaching sketching and spatial visualization skills.
Dr. Maziar Ghazinejad University of California, San Diego [biography]

Maziar Ghazinejad is an Associate teaching professor in Mechanical and Aerospace Engineering Department at UC San Diego. He received his Ph.D. in mechanical engineering from UC Riverside in 2012 and holds M.S. degrees in mechanical and electrical engineer

Download paper (2.57 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session