2026 ASEE Annual Conference & Exposition

This work presents the development and behavioral evaluation of an AI agent acting as a Virtual Teaching Assistant (VTA) for large STEM courses. The VTA is designed to engage students using Socratic dialogue and inquiry-based learning rather than direct-answer responses. The project emphasizes scalable, open-source technology that addresses challenges posed by large class sizes and limited instructor availability.
The VTA follows a multi-agent design. Its core Socratic Dialogue agent is built on Qwen2.5-7B-Instruct [1], an open-weight base model, fine-tuned with Low-Rank Adaptation (LoRA) on a domain-specific Socratic dialogue dataset that we generated by adapting the SocraticLM framework [2] to university-level thermo-fluid sciences. A planned Scaffolder agent will handle orchestration in subsequent work, including retrieval-augmented generation (RAG) over course-specific content, dialogue routing, and progress tracking. The system can be embedded in learning management systems such as Canvas or Blackboard.
Student interactions are anonymized, and summary reports can be generated for students themselves to provide on-demand feedback. Summary reports can also be generated for the course instructor to better inform them of potential gaps in understanding among the students.
This work presents the technical development and behavioral evaluation of the VTA itself. Planned future work will pilot the VTA in multiple course sections and conduct mixed-methods evaluation of pedagogical effectiveness, learning outcomes, and student perceptions relative to traditional human-TA support.