2025 ASEE Annual Conference & Exposition

Background
This CAREER project focuses on examining how cultural familiarity plays a role in racially minoritized students’ experience with engineering classroom assessments to advance knowledge on the fairness of such assessments. There are continual achievement gaps between racially minoritized students and racial majority students in engineering education assessments. In engineering classrooms, exams and quizzes make up a large percent of students’ course grades. Thus, student performances differences on these assessments impact their GPAs, which impact vital educational and career decisions such as selections of students to participate in internships, co-ops, research opportunities and even hiring. It is currently unknown how much of the achievement gaps reflects assessment bias as little attention has been paid on evaluating fairness of engineering education assessments. Unless more discourse on engineering education assessments’ fairness emerge to reveal underlying bias towards racially minoritized students, the field of engineering will continue to turn talented students away and its current issue of unequal representation will remain.

Purpose
This CAREER project specifically examines commonly used concept inventories (CI) in engineering classrooms and how the CI items function for racially minoritized and racial majority students. The purpose of this poster paper is to provide an overview of this CAREER research project and plan. More specifically, this project investigates three research questions: 1) To what extent do items from commonly used engineering CIs demonstrate acceptable functioning (in terms of difficulty and discrimination) for racially minoritized students when compared to racial majorities? 2) What are patterns of cultural familiarity and content of problematic items and items that show acceptable functioning? And 3) How do racially minoritized students experience testing in engineering classrooms?

Methodology/approach
This project adopts a mixed-method research design, consisting of first a quantitative phase followed by a qualitative phase. In the quantitative phase, we use Classical Test Theory to evaluate CI item functioning (in terms of difficulty and discrimination) for different racial/ethnicity groups and identify problematic items that consistently perform poorly to assess racially minoritized students. In the qualitative phase, we investigate the context of the problematic items to reveal underlying cultural familiarity bias. Also in the qualitative phase, we conduct interviews with racially minoritized students to explore their experiences with engineering classroom tests and their cognitive processes while answering the problematic CI items.

Implication
Examining the fairness of engineering education assessments addresses an overlooked research area. Revealing underlying biases in commonly used CIs promotes the fair and equitable assessment for racially minoritized students. Ultimately, this project complements other efforts of increasing diversity and inclusion in engineering education, contributing to fixing systemic issues in this field that hinders the success of underrepresented minority students.