2024 ASEE Annual Conference & Exposition

Optimizing Database Query Learning: A Generative AI Approach for Semantic Error Feedback

Presented at Teaching with ML and Generative AI

Prior research has highlighted the significant challenges that students face when navigating database systems, particularly in mastering SQL and NoSQL query languages. These challenges typically fall into two categories: syntax errors and semantic errors. Syntax errors occur when a student's query violates the grammatical rules of the SQL or NoSQL language, resulting in queries that the database system cannot execute. On the other hand, semantic errors arise when a query is syntactically correct but does not produce the expected result because it does not accurately reflect the student's intention or understanding of the data. Numerous common error types and overarching learning hurdles have been identified among learners, with a predominant focus on syntax errors in previous research.
However, there has been a noticeable gap in the study and categorization of semantic errors, which are equally critical for students’ learning and proficiency in database systems. Our study aims to fill this gap and contribute to the educational domain by significantly improving the precision and efficiency in identifying semantic errors in student submissions of SQL and NoSQL queries. We strive to achieve this by integrating the advanced capabilities of a Generative Pre-Trained Transformer (GPT) model with an existing feedback system, enhancing both the accuracy and effectiveness of error detection. We have utilized diverse datasets of student submissions, which were employed to fine-tune our GPT models. This tailored training process has enabled the models to better recognize and highlight semantic errors, while simultaneously providing constructive and meaningful feedback. The GPT models, through this customized training, have developed a deeper understanding of common student errors, leading to notable improvements in error detection accuracy and feedback quality.
Preliminary results from our research are highly encouraging, demonstrating significant advancements and highlighting the potential of large language models in database learning. By integrating these state-of-the-art computational tools into the learning environment, our study lays the groundwork for the creation of intelligent systems that offer nuanced and context-aware feedback. Such systems have the potential to significantly alleviate the learning obstacles associated with database systems, thereby enhancing the educational experience and support available to students

Authors

Abdulrahman AlRabah University of Illinois Urbana-Champaign [biography]

Abdulrahman AlRabah is a Master of Science (M.S.) in Computer Science student at the University of Illinois at Urbana-Champaign. He holds a Graduate Certificate in Computer Science from the same institution and a Bachelor of Science in Mechanical Engineering from California State University, Northridge. He has experience in various industries and has served in multiple roles throughout his professional career, including in oil and gas and co-founding a food & beverage company. His research focuses on identifying semantic errors of students and optimizing AI feedback using customized large language models (LLMs) through fine-tuning. Abdulrahman has worked on enhancing AI-based feedback models using both open and closed source models. His work aims to improve SQL teaching methodologies and develop tools that integrate machine learning concepts to enhance both student learning and instructor teaching experiences in computer education.
Sophia Yang http://orcid.org/0000-0003-1274-4851 University of Illinois Urbana-Champaign [biography]

Sophia Yang is a second-year Ph.D. candidate with research work focused in the areas of Computing Education, Database Systems, Bioinformatics algorithms, Human-Computer Interaction, and interesting intersections of the above. Current research work focuses on quantitatively and qualitatively studying how students learn SQL by utilizing Bioinformatics alignment algorithms, aiming to create instructor and student-facing tools. Her work aims to better improve students’ learning and instructors’ teaching experiences in large-scale database courses.
Dr. Abdussalam Alawini University of Illinois Urbana-Champaign [biography]

Abdussalam Alawini, a Teaching Associate Professor at the University of Illinois at Urbana-Champaign, combines an extensive industry background with academic achievements. Starting with a Computer Science degree from the University of Tripoli, he spent over six years in the tech industry before earning master's degrees in Computer Science and Engineering and Technology Management from Portland State University, where he also completed his Ph.D. His doctoral work focused on enhancing file-based dataset management for scientists. Dr. Alawini's research spans databases, applied machine learning, and educational technology, aiming to improve classroom experiences and develop advanced data management systems.

Download paper (1.78 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session