2025 ASEE Annual Conference & Exposition

Data-Driven Insights into AI-Powered Learning: Analyzing Student Interactions with AI-bot in Engineering Education

The integration of Generative Artificial Intelligence (GenAI) technologies in engineering education offers a significant opportunity to enhance students' learning experiences across various academic levels. To explore this potential, we analyzed data collected from AI-bot, a system that allows instructors to efficiently utilize a Large Language Model (LLM) on course-specific material. The resulting model is provided to students, enabling them to ask questions directly related to the course content. A unique aspect of the system is its ability to restrict responses strictly to the course material provided to it, ensuring accuracy and relevance. Additionally, the system logs all student prompts and model responses, creating a rich dataset for studying student interactions with the AI. Our findings show that some courses had more queries related to deeper conceptual understanding, while others focused more on logistical queries, highlighting the AI-bot’s flexibility in meeting diverse student needs. This work contributes to the growing field of AI-assisted education by presenting a practical implementation of GenAI and offering insights into optimizing its application in academic settings.

Authors

Abdulrahman AlRabah University of Illinois at Urbana - Champaign [biography]

Abdulrahman AlRabah holds a Master of Science (M.S.) in Computer Science from the University of Illinois at Urbana-Champaign and a Bachelor of Science in Mechanical Engineering from California State University, Northridge. With experience across various industries, including oil and gas and co-founding a food & beverage company, his research focuses on optimizing AI feedback through customized large language models (LLMs) and improving computing courses to enhance both student learning and instructor experiences.
Zepei Li University of Illinois at Urbana Champaign [biography]

Zepei Li is a Ph.D. student in Computer Science at the University of Illinois Urbana-Champaign (UIUC). He holds an M.S. and B.S. in Computer Science from UIUC. His research interests span AI in education, computer science education, and human-computer interaction. His previous work includes assessing student learning in database query languages and examining students' behaviors, attitudes, and beliefs toward generative AI in educational settings. He aims to enhance AI tools to better support educational applications.
Ms. Meredith Blumthal University of Illinois at Urbana - Champaign [biography]

Meredith Blumthal has been in the field of international education for 15 years. As the Director for International Programs in Engineering (IPENG) at the University of Illinois, she leads the study abroad initiatives and programming for the college. Ms. Blumthal’s team includes three study abroad advisors, a receptionist and peer advisors. Together the IPENG office provides study abroad advising, expertise, international exchanges, and cultural experiences in over 60 programs throughout 20 countries, with the goal of providing transformative learning experiences that empower students to develop global competency skills. Through partnerships with partner universities abroad, alumni, faculty, staff and student groups, Ms. Blumthal leads the IPENG team to provide study abroad advising, informational events, and class presentations. She works with the 12 engineering departments to develop and grow studying abroad programming with overseas partners to grow connections with universities, and facilitates the study abroad fair. Meredith is responsible for developing faculty-led programing an engagement within the College of Engineering.
Her areas of expertise include ideation, program development, partner collaboration, faculty training and development, comprehensive program management, marketing and promotion, co-curricular development, study abroad advising, and student development.
Dr. Sotiria Koloutsou-Vakakis http://orcid.org/0000-0002-7658-6517 University of Illinois Urbana-Champaign [biography]

Dr. Sotiria Koloutsou-Vakakis holds a Diploma in Surveying Engineering (National Technical University of Athens, Greece), a M.A. in Geography (University of California, Los Angeles), and M.S. and Ph.D. degrees in Environmental Engineering (University of Illinois Urbana-Champaign). She is a Senior Lecturer and Research Scientist in the Department of Civil and Environmental Engineering, at the University of Illinois Urbana-Champaign. Her main interests are in air quality, environmental policy, and supporting student learning and professional preparation.
Dr. Volodymyr Kindratenko University of Illinois Urbana-Champaign [biography]

Volodymyr Kindratenko is the Director for the Center for Artificial Intelligence Innovation (CAII) at the National Center for Supercomputing Applications (NCSA). He is also an Adjunct Associate Professor in the Departments of Electrical and Computer Engineering and a Research Associate Professor in the Siebel School of Computing and Data Science. His research interests lie at the intersection of AI systems and applications, with a focus on generative AI solutions for science and education. His team is developing the uiuc.chat chatbot platform, which facilitates rapid knowledge distillation.
Prof. Tomasz Kozlowski http://orcid.org/0000-0001-9317-5309 University of Illinois Urbana-Champaign [biography]

Dr. Tomasz Kozlowski is a Professor and Associate Head for the Undergraduate Programs in the Department of Nuclear, Plasma, and Radiological Engineering at the University of Illinois Urbana-Champaign. He received a Ph.D. in Nuclear Engineering from Purdue University in 2005. He has been an active member of the American Nuclear Society (since 1997) and the American Society for Engineering Education (since 2016). He is also a member of the OECD/NEA Nuclear Science Committee Working Party on Reactor Systems (NSC/WPRS) (since 2010) and is serving on the Scientific Board and Technical Committee of the OECD/NEA Benchmark for Uncertainty Analysis in Best-Estimate Modelling for Design, Operation and Safety Analysis of Light Water Reactors (LWR-UAM) (since 2008).
Dr. Abdussalam Alawini University of Illinois Urbana - Champaign [biography]

Dr. Abdussalam Alawini is a teaching associate professor in the Department of Computer
Science at the University of Illinois Urbana-Champaign. His research interests are broadly in
database systems and computing education. In particular, he is interested in the application of
AI technologies in facilitating active learning activities and improving online assessments. Dr. Alawini
is very passionate about teaching. His teaching excellence is evidenced by his consistent
recognition in the 'Excellent Teachers Ranked by their Students' over several semesters and his
receipt of the 2024 Campus Excellence in Undergraduate Teaching award.

Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 22, 2025, and to all visitors after the conference ends on June 25, 2025