2026 ASEE Annual Conference & Exposition

Writing-to-Learn (WTL) activities use open-ended prompts to help students articulate understanding, reflect on learning strategies, and make their thinking visible. However, the increasing availability of generative AI tools has created a validity challenge for written WTL artifacts, because students may produce polished written responses without fully engaging in their own reasoning. This study explores a complementary approach based on time-windowed oral responses, in which students verbalize their thinking in response to open-ended WTL prompts. The approach is designed to preserve the reflective purpose of WTL while reducing opportunities for AI-generated substitution.

A cloud-based oral-response pipeline was developed to collect students’ time-constrained responses, generate automatic transcripts, and make both audio and text available for analysis. The pipeline was deployed in two undergraduate engineering courses to collect responses associated with learning plans, learning evaluations, and learning reflections. This study focuses on a necessary first step: evaluating whether automatically generated transcripts accurately preserve students’ spoken responses and intended meaning.

Transcription fidelity was evaluated using word error rate, character error rate, semantic similarity based on embedding representations, error-type analysis, and student self-reported transcript accuracy ratings. Results from 19 students and 51 rated oral-response submissions showed a word error rate of 7.85%, a character error rate of 3.78%, and an average semantic similarity score of 0.93. Student ratings also indicated generally positive perceptions of transcript accuracy, with 82.4% of rated submissions receiving four or five stars.

These findings suggest that time-windowed oral responses can be captured and transcribed with sufficient fidelity to support further analysis within WTL contexts. The study establishes a foundation for integrating transcribed oral responses into future AI-powered, rubric-based assessment pipelines that can evaluate cognitive, metacognitive, and motivational dimensions of student learning, identify learning strengths and weaknesses, and support adaptive feedback in the era of generative AI.