2026 ASEE Annual Conference & Exposition

As artificial intelligence (AI) tools become increasingly integrated into educational research and assessment, their growing influence raises critical questions about interpretation, context, and meaning. While AI systems excel at identifying quantitative patterns, they are limited in matching the human capacity to recognize nuance, developmental differences, and socio-emotional context that are critical factors within complex learning environments. This paper examines these limitations through a case study that compares AI-generated and human-researcher analyses of outcomes in an engineering summer bridge program.

The study highlights how AI approaches data through precision, standardization, and statistical reasoning, often framing findings in terms of power, reliability, or numerical relationships. In contrast, human interpretation emphasizes the lived experiences behind the data—how students’ developmental stages, prior experiences, and personal reflections shape their engagement with mentorship and identity formation. By juxtaposing these analytical perspectives, the paper explores the complementary strengths and weaknesses of each approach and illustrates the importance of hybrid methodologies that combine computational efficiency with contextual understanding.

Serving as a case study, this work examines a six-week summer bridge program for pre-college and first-year engineering students. Using pre- and post-survey data, both AI analytical models and human researchers analyzed patterns in mentorship, engineering identity, and early retention intentions to reveal how differing interpretive lenses shape conclusions about student experience and program impact.

Ultimately, this paper argues that as AI becomes increasingly embedded in educational research, critical reflection is needed to ensure that data-driven insights do not eclipse human meaning. The case study underscores AI’s strengths in precision, consistency, and rapid quantitative synthesis, but also its weaknesses in perceiving developmental nuance, contextual meaning, and socio-emotional growth. Together, these perspectives illustrate the necessity of hybrid human–AI research approaches that integrate computational rigor with contextual interpretation—ensuring that data-driven analyses in engineering education capture not only statistical patterns but also the human experiences they represent.