2026 ASEE Annual Conference & Exposition

Evaluating the Reliability of Out-of-Class Assessments Amid Widespread Use of AI

Presented at Mechanical Engineering (MECH) Session 5: Educational Frameworks, Pedagogy, and Learning Systems

Assessing student knowledge in engineering courses has become increasingly challenging due to widespread access to external resources. Homework assignments and take-home assessments, which are often used to evaluate higher-level problem-solving skills, have long been vulnerable to solution manuals and online tutoring services. In upper-level undergraduate and graduate engineering courses, the complexity of problems frequently necessitates assessment formats that extend beyond standard in-class exam periods, further increasing reliance on take-home exams and projects.

More recently, the availability of artificial intelligence (AI)–based tools have introduced additional concerns regarding the reliability of take-home assessments. AI tools can generate complete and well-structured solutions to complex engineering problems, making it difficult to determine the extent to which submitted work reflects individual student understanding. This paper examines the impact of AI use on the assessment of student learning in undergraduate and graduate mechanical engineering courses.

Examples of assigned problems and corresponding AI-generated solutions are presented to illustrate cases in which AI produces correct, partially correct, or incorrect results. In addition, results from an anonymous student survey are analyzed to examine how students are using AI tools. Survey findings indicate that students increasingly view AI as a tutoring resource to support learning, while also acknowledging its use as a shortcut to obtaining problem solutions, which may limit learning. These results provide insight into both the opportunities and challenges AI presents for engineering assessment.

Authors

Dr. Amir Karimi The University of Texas at San Antonio [biography]

Amir Karimi, University of Texas, San Antonio Amir Karimi is a Professor of Mechanical Engineering at The University of Texas at San Antonio (UTSA). He received his Ph.D. degree in Mechanical Engineering from the University of Kentucky in 1982. His teaching and research interests are in thermal sciences. He has served as the Chair of Mechanical Engineering (1987 to 1992 and September 1998 to January of 2003), College of Engineering Associate Dean of Academic Affairs (Jan. 2003-April 2006), and the Associate Dean of Undergraduate Studies (April 2006-September 2013). Dr. Karimi is a Fellow of ASEE, a Fellow of ASME, senior member of AIAA, and holds membership in ASHRAE, and Sigma Xi. He has served as the ASEE Campus Representative at UTSA, ASEE-GSW Section Campus Representative, and served as the Chair of ASEE Zone III (2005-07). He chaired the ASEE-GSW section during the 1996-97 academic year.
Dr. Randall D. Manteufel http://orcid.org/0000-0001-9829-2723 The University of Texas at San Antonio [biography]

Dr. Randall Manteufel is an Associate Professor of Mechanical Engineering at The University of Texas at San Antonio (UTSA). He has won several teaching awards, including the 2012 University of Texas System Regents Outstanding Teaching Award and the 2013 UTSA President’s Distinguished Achievement Award for Teaching Excellence, the 2010, 2014, 2018 and 2019 College of Engineering Student Council Professor of the Year Award, 2008, 2022, 2024 College Excellence in Teaching, and 2005 Mechanical Engineering Instructor of the year award, 1999 ASEE-GSW Outstanding New Faculty Award. His teaching and research interests are in the thermal sciences. In 2015-2016, he chaired the American Society for Engineering Education Gulf Southwest section and in 2018-2019 he chaired the Academy of Distinguished Teaching Scholars at UTSA. He is a registered Professional Engineer in Texas.
Ender A Finol http://orcid.org/0000-0002-0811-9482 The University of Texas at San Antonio [biography]

Dr. Ender Finol is a Professor and the Zachry Endowed Department Chair of Mechanical Engineering at University of Texas at San Antonio. He teaches the courses Thermodynamics I and Fluid Mechanics as part of the undergraduate core curriculum in mechanical engineering. His research expertise is in vascular biofluid mechanics and soft tissue mechanics.

Download paper (1.51 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session