2025 ASEE Annual Conference & Exposition

Assessing the Impact of the Use of Generative AI in Developing and Using Assessment Grading Rubrics for Engineering Courses

Presented at Generative AI and Its Role in Industrial Engineering

Engineering education is rapidly integrating generative artificial-intelligence (GenAI) tools that promise faster, more consistent assessment—yet their reliability in discipline-specific contexts remains uncertain. This mixed-methods study compared ChatGPT-4, Claude 3.5, and Perplexity AI across four undergraduate engineering assignments (two lower-level, two upper-level). Quantitative analyses - one-way ANOVA followed by Tukey’s HSD (α = .05) contrasted AI scores with expert grades, while qualitative feedback from faculty and students captured perceptions of clarity, fairness, and workload. ChatGPT-4 mirrored expert grades on complex tasks (|Δ| ≤ 3.5 %), whereas Claude 3.5 and Perplexity AI under-scored upper-level work by as much as 27 %. Stakeholders appreciated the rubric’s consistency and faster turnaround but criticized the models’ rigidity and opaque rationales. These findings support a hybrid approach in which AI tools provide baseline scores and instructors supply higher-order judgement. Further research should examine discipline-specific fine-tuning and the long-term impact of AI-assisted grading on student learning and educator workload.

Authors

Dr. THOMAS AMING'A OMWANDO Simpson University [biography]

Dr. Thomas Omwando holds a PhD in Industrial Engineering from the University of Wisconsin Milwaukee. He is the Chair and Associate Professor of Engineering at Simpson University and his teaching interests are in statistical quality control, engineering/project management, engineering economy, capstone design and production and operations analysis. His research interests are in sustainable manufacturing, entrepreneurially minded learning, project based learning, and the use of generative AI in enhancing engineering education education
Dr. Adel Alhalawani Rose-Hulman Institute of Technology [biography]

Dr. Adel Alhalawani’s teaching interests include engineering design, biomechanics, statics and mechanics of materials, biomaterials and design of manufacturing. His research has focused on bio-glass based adhesives for orthopedic applications and dental-based adhesives.
Dr. Ashutosh Khandha http://orcid.org/0000-0001-6003-3881 University of Delaware [biography]

Ashutosh Khandha is an Assistant Professor in the Biomedical Engineering Department at the University of Delaware, with work experience both in Industry and Academia. His interests and expertise pertain to Engineering Education and Accreditation, Forensic Biomechanics, Medical Devices, Clinical Imaging and Bioinstrumentation.
Dr. Bhavana Kotla http://orcid.org/0000-0002-5163-7664 The Ohio State University [biography]

Visiting Assistant Professor, Department of Engineering Education, College of Engineering, The Ohio State University

Download paper (1.7 MB)

» Download paper

« View session