2026 ASEE Annual Conference & Exposition

AI in Teaching: Evaluating Open-Ended Projects Using an AI Chatbot

Presented at Mechanical Engineering (MECH) Session 2: AI and Emerging Technologies in Mechanical Engineering Education

A senior-level Mechatronics Design course (MEE4177) was launched in Fall 2025 in the Department of Mechanical Engineering at Temple University. The course was organized around the 2025 to 2026 ASME Student Design Competition, in which devices for collecting, sorting, and delivering colored cube-shaped waste were designed, fabricated, and tested. Work spanned mechanical gantry design, motion control, sensing, machine vision, and embedded programming, and students were assigned distinct focus areas.
To support evaluation in this open-ended setting, a custom AI Chatbot was developed and trained on course materials, prior project reports, and instructor prompts. The chatbot was used as a personalized, supportive reviewer that conducted conversational evaluations aligned to each student’s chosen subsystem. Direct feedback was provided, probing questions were posed, and hints to relevant engineering principles were offered when students were stuck. In this way, pressure free engagement was encouraged and personalized evaluation across diverse focus areas was enabled.
Chat histories were recorded and a follow up survey was administered to gather qualitative perceptions of fairness, engagement, and validity. Lessons learned are reported, including feasibility for assessing participation and mastery and practical considerations for integrating AI assisted evaluation in design courses. Evidence suggests that conversational assessment can complement traditional methods in engineering education.

Authors
Download paper (839 KB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.