2025 ASEE Annual Conference & Exposition

First-year ECE students from Institution A and Institution B worked remotely on an inter-university team design project. The project was implemented in Spring 2023 and repeated in Spring 2024. At the end of the endeavor, the students completed an end of project survey and wrote a reflection about the experience.

Following the initial project offering, the authors employed Natural Language Processing (NLP) techniques to analyze the student reflections. Two unsupervised learning techniques (K-means clustering and Latent Dirichlet Allocation (LDA) with Non-Negative Matrix Factorization) were utilized to identify key themes in the student responses and categorize the topics or themes common among the responses. Preliminary findings based on the Spring 2023 data revealed a set of five common and distinctive themes or topics (performance expectations, collaboration and planning, skill development, problem-solving, and evaluation) across the reports from both institutions and were reported by the authors in a previous publication.

Building on this initial work, the authors repeated the analysis techniques on the data collected during the subsequent project offering. Additionally, the authors used the themes identified from the initial offering to train a classifier. The classifier was used to label and categorize the student reflections from the second cohort based on the themes uncovered or “learned” when analyzing the first cohort of responses.

Through replicating the previously reported analyses as well as using the previous work as the training data set for labeling the results from subsequent project offerings, the authors gained insight into the validity of the techniques and the effectiveness of using unsupervised NLP methods to uncover insights from open-ended student responses and reflections. In applying various methods and validating their efficacy, the authors gained valuable insight into the possibility for NLP and other AI-assisted techniques to be used for academic assessments and for labeling responses as part of qualitative, or mixed-methods educational research endeavors.

The paper will detail the application of the techniques, including the necessary preprocessing, to unstructured text collected as free-response student reflections. Lessons learned about the process of applying the techniques as well as insights gained about the student experience as captured in their reflections will be reported. Recommendations for the use of the AI-assisted process to analyze qualitative data to better measure outcomes and student sentiments regarding a course lesson or experience will be reported.