2026 ASEE Annual Conference & Exposition

AI as a Constrained Guide: Measuring Knowledge Transfer in a Graduate Engineering Course

Presented at Electrical and Computer Engineering Division (ECE) Technical Session 4

This paper presents a novel pedagogical experiment in a graduate-level engineering course, designed to assess how students integrate newfound knowledge and expertise when utilizing conversational AI as a constrained project assistant. Our primary goal was to move beyond the typical "AI as a tutor" model to establish the AI as an effective "guide," requiring students to generate content from their own knowledge base while using the AI solely for structural critique, redirection, and resource navigation. The core of the study involved a structured, two-phase project proposal exercise. Early in the semester, before formal introduction to core course topics, students drafted an initial proposal for a physics-aware deep learning project, leveraging a conversational AI assistant whose information output was programmatically limited (e.g., restricted to providing only conceptual definitions or structural feedback). A second, identical proposal was drafted at the end of the semester after all technical topics had been covered. We hypothesize that the knowledge gained throughout the semester would significantly improve the quality and technical depth of the final proposal. More critically, we aim to decouple the effect of knowledge gained (understanding concepts) from expertise gained (skilled interaction with the AI tool). The evaluation utilizes a mixed-methods approach. Quantitative metrics are derived from the students' tracked interactions with the conversational AI. Qualitative and self-reported measures are collected via surveys at the end of each assignment, which capture their evolving confidence in using the AI as an assistant rather than a solution provider.

Authors

Dr. Edgar Lobaton http://orcid.org/0000-0002-4056-8309 North Carolina State University at Raleigh [biography]

Edgar J. Lobaton is a Professor in the Department of Electrical and Computer Engineering (ECE) at North Carolina State University (NCSU). He joined the department in 2011. Lobaton earned his B.S. in Mathematics and Electrical engineering from Seattle University in 2004. He completed his Ph.D. in Electrical Engineering and Computer Sciences from the University of California, Berkeley in 2009. Lobaton was engaged in research at Alcatel-Lucent Bell Labs in 2005 and 2009. He was awarded the NSF CAREER Award in 2016. He was also awarded the 2009 Computer Innovation Fellows post-doctoral fellowship and conducted research in the Department of Computer Science at the University of North Carolina (UNC) at Chapel Hill from 2009 until 2011. In 2024, he received the University Faculty Scholars and the Outstanding Teacher Awards from NC State. His research focuses on the integration of AI, and physical and probabilistic modeling applied to cyber-physical systems in areas such as wearable health monitoring, rehabilitation robotics, agriculture and biological imaging.
Mayur Sanap North Carolina State University at Raleigh [biography]

Mayur Sanap is a Ph.D. student in Electrical Engineering at North Carolina State University. His research interests include edge AI, AI for healthcare, and embedded systems. In this paper, he contributed by developing the AI metrics and conducting a literature review of prior work.
Ms. Ishita Pai Raikar North Carolina State University at Raleigh [biography]

Ishita Pai Raikar is a Computer Engineering student at North Carolina State University with a minor in Philosophy. Her research focuses on AI ethics, human-robot interaction, and responsible AI integration in educational and social systems. She works with the NeuroComputational Ethics Research Group, where she studies ethical frameworks for socially assistive robotics and the moral dimensions of AI-mediated decision-making. She also leads AI Revolution: an #EthicsQuest Across Disciplines, a panel series that brings together academic, industry, and policy experts to examine the future of human-centered and ethically aligned AI. Her broader interests include AI-enhanced learning, interdisciplinary collaboration, and the societal impacts of emerging technologies.
Dr. Carine Marette Indiana University-Bloomington [biography]

Carine Marette is a doctoral candidate in Instructional System Technology at Indiana University Bloomington's School of Education, specializing in AI-enhanced engineering pedagogy and learning analytics. Her research examines how constrained artificial intelligence environments can function as measurement affordances for observing transfer-relevant behaviors in authentic learning contexts. She developed the Vector Framework for multi-source learning telemetry, integrating behavioral data from AI interaction logs and peer assessment platforms to measure transfer readiness, productive struggle, and metacognitive development. Her work bridges engineering education, cognitive load theory, and educational measurement, with particular focus on developing validity arguments for AI-mediated assessment in project-based learning environments. Carine co-created with Kritik360 and VisibleAI platforms, which provide transparency-oriented authorship tracking and structured peer evaluation systems. Her dissertation investigates adaptive constraint design for differentiated AI scaffolding across learner distributions in graduate engineering courses.
Md Halim Mondol North Carolina State University at Raleigh [biography]

Md Halim Mondol is a PhD student in the Department of Electrical and Computer Engineering at North Carolina State University. His research centers on machine learning, large language models, and generative models, particularly their application to knowledge hub development and cheminformatics, aiming to advance data-driven discovery and intelligent information systems.

Download paper (1.36 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session